* [PATCH v2 00/34] Convert RPC client transmission to a queued model
@ 2018-09-04 21:05 Trond Myklebust
2018-09-04 21:05 ` [PATCH v2 01/34] SUNRPC: Clean up initialisation of the struct rpc_rqst Trond Myklebust
0 siblings, 1 reply; 45+ messages in thread
From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw)
To: linux-nfs
For historical reasons, the RPC client is heavily serialised during the
process of transmitting a request by the XPRT_LOCK. A request is
required to take that lock before it can start XDR encoding, and it is
required to hold it until it is done transmitting. In essence the lock
protects the following functions:
- Stream based transport connect/reconnect
- RPCSEC_GSS encoding of the RPC message
- Transmission of a single RPC message
The following patch set assumes that we do not need to do much to
improve performance of the connect/reconnect case, as that is supposed
to be a rare occurrence.
The set looks at dealing with RPCSEC_GSS issues by removing serialisation
while encoding, and simply assuming that if we detect after grabbing the
XPRT_LOCK that we're about to transmit a message with a sequence number
that has fallen outside the window allowed by RFC2203, then we can
abort the transmission of that message, and schedule it for re-encoding.
Since window sizes are typically expected to lie above 100 messages or
so, we expect these cases where we miss the window to be rare, in
general.
Finally, we look at trying to avoid the requirement that every request
must go through the process of being woken up to grab the XPRT_LOCK in
order to transmit itself by allowing a request that currently holds the
XPRT_LOCK to grab other requests from an ordered queue, and to transmit
them too. The bulk of the changes in this patchset are dedicated to
providing this functionality.
In addition, the XPRT_LOCK queue provides some extra functionality:
- Throttling of the TCP slot allocation (as Chuck pointed out)
- Fair queuing, to ensure batch jobs don't crowd out interactive ones
The patchset does add functionality to ensure that the resulting
transmission queue is fair, and also fixes up the RPC wait queues to
ensure that they don't compromise fairness.
For now, this patchset discards the TCP slot throttling. We may still
want to throttle in the case where the connection is lost, but if we
do so, we should ensure we do not serialise all requests when in the
connected state.
---
v2: - Address feedback by Chuck.
- Handle UDP/RDMA credits correctly
- Remove throttling of TCP slot allocations
- Minor nits
- Clean up the write_space handling
- Fair queueing
Trond Myklebust (34):
SUNRPC: Clean up initialisation of the struct rpc_rqst
SUNRPC: If there is no reply expected, bail early from call_decode
SUNRPC: The transmitted message must lie in the RPCSEC window of
validity
SUNRPC: Simplify identification of when the message send/receive is
complete
SUNRPC: Avoid holding locks across the XDR encoding of the RPC message
SUNRPC: Rename TCP receive-specific state variables
SUNRPC: Move reset of TCP state variables into the reconnect code
SUNRPC: Add socket transmit queue offset tracking
SUNRPC: Simplify dealing with aborted partially transmitted messages
SUNRPC: Refactor the transport request pinning
SUNRPC: Add a helper to wake up a sleeping rpc_task and set its status
SUNRPC: Don't wake queued RPC calls multiple times in xprt_transmit
SUNRPC: Rename xprt->recv_lock to xprt->queue_lock
SUNRPC: Refactor xprt_transmit() to remove the reply queue code
SUNRPC: Refactor xprt_transmit() to remove wait for reply code
SUNRPC: Minor cleanup for call_transmit()
SUNRPC: Distinguish between the slot allocation list and receive queue
NFS: Add a transmission queue for RPC requests
SUNRPC: Refactor RPC call encoding
SUNRPC: Treat the task and request as separate in the
xprt_ops->send_request()
SUNRPC: Don't reset the request 'bytes_sent' counter when releasing
XPRT_LOCK
SUNRPC: Simplify xprt_prepare_transmit()
SUNRPC: Move RPC retransmission stat counter to xprt_transmit()
SUNRPC: Fix up the back channel transmit
SUNRPC: Support for congestion control when queuing is enabled
SUNRPC: Improve latency for interactive tasks
SUNRPC: Allow calls to xprt_transmit() to drain the entire transmit
queue
SUNRPC: Queue the request for transmission immediately after encoding
SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue
SUNRPC: Allow soft RPC calls to time out when waiting for the
XPRT_LOCK
SUNRPC: Turn off throttling of RPC slots for TCP sockets
SUNRPC: Clean up transport write space handling
SUNRPC: Cleanup: remove the unused 'task' argument from the
request_send()
SUNRPC: Queue fairness for all.
include/linux/sunrpc/auth.h | 2 +
include/linux/sunrpc/auth_gss.h | 1 +
include/linux/sunrpc/sched.h | 9 +-
include/linux/sunrpc/svc_xprt.h | 1 -
include/linux/sunrpc/xprt.h | 31 +-
include/linux/sunrpc/xprtsock.h | 23 +-
include/trace/events/sunrpc.h | 10 +-
net/sunrpc/auth.c | 10 +
net/sunrpc/auth_gss/auth_gss.c | 41 ++
net/sunrpc/backchannel_rqst.c | 4 +-
net/sunrpc/clnt.c | 152 ++---
net/sunrpc/sched.c | 189 +++---
net/sunrpc/svc_xprt.c | 2 -
net/sunrpc/svcsock.c | 6 +-
net/sunrpc/xprt.c | 679 +++++++++++++--------
net/sunrpc/xprtrdma/backchannel.c | 7 +-
net/sunrpc/xprtrdma/rpc_rdma.c | 12 +-
net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 14 +-
net/sunrpc/xprtrdma/transport.c | 10 +-
net/sunrpc/xprtsock.c | 359 ++++++-----
20 files changed, 919 insertions(+), 643 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 45+ messages in thread* [PATCH v2 01/34] SUNRPC: Clean up initialisation of the struct rpc_rqst 2018-09-04 21:05 [PATCH v2 00/34] Convert RPC client transmission to a queued model Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 02/34] SUNRPC: If there is no reply expected, bail early from call_decode Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Move the initialisation back into xprt.c. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 1 - net/sunrpc/clnt.c | 1 - net/sunrpc/xprt.c | 91 +++++++++++++++++++++---------------- 3 files changed, 51 insertions(+), 42 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 336fd1a19cca..3d80524e92d6 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -325,7 +325,6 @@ struct xprt_class { struct rpc_xprt *xprt_create_transport(struct xprt_create *args); void xprt_connect(struct rpc_task *task); void xprt_reserve(struct rpc_task *task); -void xprt_request_init(struct rpc_task *task); void xprt_retry_reserve(struct rpc_task *task); int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task); int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task); diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 8ea2f5fadd96..bc9d020bf71f 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1558,7 +1558,6 @@ call_reserveresult(struct rpc_task *task) task->tk_status = 0; if (status >= 0) { if (task->tk_rqstp) { - xprt_request_init(task); task->tk_action = call_refresh; return; } diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index a8db2e3f8904..6aa09edc9567 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1250,6 +1250,55 @@ void xprt_free(struct rpc_xprt *xprt) } EXPORT_SYMBOL_GPL(xprt_free); +static __be32 +xprt_alloc_xid(struct rpc_xprt *xprt) +{ + __be32 xid; + + spin_lock(&xprt->reserve_lock); + xid = (__force __be32)xprt->xid++; + spin_unlock(&xprt->reserve_lock); + return xid; +} + +static void +xprt_init_xid(struct rpc_xprt *xprt) +{ + xprt->xid = prandom_u32(); +} + +static void +xprt_request_init(struct rpc_task *task) +{ + struct rpc_xprt *xprt = task->tk_xprt; + struct rpc_rqst *req = task->tk_rqstp; + + INIT_LIST_HEAD(&req->rq_list); + req->rq_timeout = task->tk_client->cl_timeout->to_initval; + req->rq_task = task; + req->rq_xprt = xprt; + req->rq_buffer = NULL; + req->rq_xid = xprt_alloc_xid(xprt); + req->rq_connect_cookie = xprt->connect_cookie - 1; + req->rq_bytes_sent = 0; + req->rq_snd_buf.len = 0; + req->rq_snd_buf.buflen = 0; + req->rq_rcv_buf.len = 0; + req->rq_rcv_buf.buflen = 0; + req->rq_release_snd_buf = NULL; + xprt_reset_majortimeo(req); + dprintk("RPC: %5u reserved req %p xid %08x\n", task->tk_pid, + req, ntohl(req->rq_xid)); +} + +static void +xprt_do_reserve(struct rpc_xprt *xprt, struct rpc_task *task) +{ + xprt->ops->alloc_slot(xprt, task); + if (task->tk_rqstp != NULL) + xprt_request_init(task); +} + /** * xprt_reserve - allocate an RPC request slot * @task: RPC task requesting a slot allocation @@ -1269,7 +1318,7 @@ void xprt_reserve(struct rpc_task *task) task->tk_timeout = 0; task->tk_status = -EAGAIN; if (!xprt_throttle_congested(xprt, task)) - xprt->ops->alloc_slot(xprt, task); + xprt_do_reserve(xprt, task); } /** @@ -1291,45 +1340,7 @@ void xprt_retry_reserve(struct rpc_task *task) task->tk_timeout = 0; task->tk_status = -EAGAIN; - xprt->ops->alloc_slot(xprt, task); -} - -static inline __be32 xprt_alloc_xid(struct rpc_xprt *xprt) -{ - __be32 xid; - - spin_lock(&xprt->reserve_lock); - xid = (__force __be32)xprt->xid++; - spin_unlock(&xprt->reserve_lock); - return xid; -} - -static inline void xprt_init_xid(struct rpc_xprt *xprt) -{ - xprt->xid = prandom_u32(); -} - -void xprt_request_init(struct rpc_task *task) -{ - struct rpc_xprt *xprt = task->tk_xprt; - struct rpc_rqst *req = task->tk_rqstp; - - INIT_LIST_HEAD(&req->rq_list); - req->rq_timeout = task->tk_client->cl_timeout->to_initval; - req->rq_task = task; - req->rq_xprt = xprt; - req->rq_buffer = NULL; - req->rq_xid = xprt_alloc_xid(xprt); - req->rq_connect_cookie = xprt->connect_cookie - 1; - req->rq_bytes_sent = 0; - req->rq_snd_buf.len = 0; - req->rq_snd_buf.buflen = 0; - req->rq_rcv_buf.len = 0; - req->rq_rcv_buf.buflen = 0; - req->rq_release_snd_buf = NULL; - xprt_reset_majortimeo(req); - dprintk("RPC: %5u reserved req %p xid %08x\n", task->tk_pid, - req, ntohl(req->rq_xid)); + xprt_do_reserve(xprt, task); } /** -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 02/34] SUNRPC: If there is no reply expected, bail early from call_decode 2018-09-04 21:05 ` [PATCH v2 01/34] SUNRPC: Clean up initialisation of the struct rpc_rqst Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 03/34] SUNRPC: The transmitted message must lie in the RPCSEC window of validity Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index bc9d020bf71f..4f1ec8013332 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2260,6 +2260,11 @@ call_decode(struct rpc_task *task) dprint_status(task); + if (!decode) { + task->tk_action = rpc_exit_task; + return; + } + if (task->tk_flags & RPC_CALL_MAJORSEEN) { if (clnt->cl_chatty) { printk(KERN_NOTICE "%s: server %s OK\n", @@ -2297,13 +2302,11 @@ call_decode(struct rpc_task *task) goto out_retry; return; } - task->tk_action = rpc_exit_task; - if (decode) { - task->tk_status = rpcauth_unwrap_resp(task, decode, req, p, - task->tk_msg.rpc_resp); - } + task->tk_status = rpcauth_unwrap_resp(task, decode, req, p, + task->tk_msg.rpc_resp); + dprintk("RPC: %5u call_decode result %d\n", task->tk_pid, task->tk_status); return; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 03/34] SUNRPC: The transmitted message must lie in the RPCSEC window of validity 2018-09-04 21:05 ` [PATCH v2 02/34] SUNRPC: If there is no reply expected, bail early from call_decode Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 04/34] SUNRPC: Simplify identification of when the message send/receive is complete Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs If a message has been encoded using RPCSEC_GSS, the server is maintaining a window of sequence numbers that it considers valid. The client should normally be tracking that window, and needs to verify that the sequence number used by the message being transmitted still lies inside the window of validity. So far, we've been able to assume this condition would be realised automatically, since the client has been encoding the message only after taking the socket lock. Once we change that condition, we will need the explicit check. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/auth.h | 2 ++ include/linux/sunrpc/auth_gss.h | 1 + net/sunrpc/auth.c | 10 ++++++++ net/sunrpc/auth_gss/auth_gss.c | 41 +++++++++++++++++++++++++++++++++ net/sunrpc/clnt.c | 3 +++ net/sunrpc/xprt.c | 7 ++++++ 6 files changed, 64 insertions(+) diff --git a/include/linux/sunrpc/auth.h b/include/linux/sunrpc/auth.h index 58a6765c1c5e..2c97a3933ef9 100644 --- a/include/linux/sunrpc/auth.h +++ b/include/linux/sunrpc/auth.h @@ -157,6 +157,7 @@ struct rpc_credops { int (*crkey_timeout)(struct rpc_cred *); bool (*crkey_to_expire)(struct rpc_cred *); char * (*crstringify_acceptor)(struct rpc_cred *); + bool (*crneed_reencode)(struct rpc_task *); }; extern const struct rpc_authops authunix_ops; @@ -192,6 +193,7 @@ __be32 * rpcauth_marshcred(struct rpc_task *, __be32 *); __be32 * rpcauth_checkverf(struct rpc_task *, __be32 *); int rpcauth_wrap_req(struct rpc_task *task, kxdreproc_t encode, void *rqstp, __be32 *data, void *obj); int rpcauth_unwrap_resp(struct rpc_task *task, kxdrdproc_t decode, void *rqstp, __be32 *data, void *obj); +bool rpcauth_xmit_need_reencode(struct rpc_task *task); int rpcauth_refreshcred(struct rpc_task *); void rpcauth_invalcred(struct rpc_task *); int rpcauth_uptodatecred(struct rpc_task *); diff --git a/include/linux/sunrpc/auth_gss.h b/include/linux/sunrpc/auth_gss.h index 0c9eac351aab..30427b729070 100644 --- a/include/linux/sunrpc/auth_gss.h +++ b/include/linux/sunrpc/auth_gss.h @@ -70,6 +70,7 @@ struct gss_cl_ctx { refcount_t count; enum rpc_gss_proc gc_proc; u32 gc_seq; + u32 gc_seq_xmit; spinlock_t gc_seq_lock; struct gss_ctx *gc_gss_ctx; struct xdr_netobj gc_wire_ctx; diff --git a/net/sunrpc/auth.c b/net/sunrpc/auth.c index 305ecea92170..59df5cdba0ac 100644 --- a/net/sunrpc/auth.c +++ b/net/sunrpc/auth.c @@ -817,6 +817,16 @@ rpcauth_unwrap_resp(struct rpc_task *task, kxdrdproc_t decode, void *rqstp, return rpcauth_unwrap_req_decode(decode, rqstp, data, obj); } +bool +rpcauth_xmit_need_reencode(struct rpc_task *task) +{ + struct rpc_cred *cred = task->tk_rqstp->rq_cred; + + if (!cred || !cred->cr_ops->crneed_reencode) + return false; + return cred->cr_ops->crneed_reencode(task); +} + int rpcauth_refreshcred(struct rpc_task *task) { diff --git a/net/sunrpc/auth_gss/auth_gss.c b/net/sunrpc/auth_gss/auth_gss.c index 21c0aa0a0d1d..c898a7c75e84 100644 --- a/net/sunrpc/auth_gss/auth_gss.c +++ b/net/sunrpc/auth_gss/auth_gss.c @@ -1984,6 +1984,46 @@ gss_unwrap_req_decode(kxdrdproc_t decode, struct rpc_rqst *rqstp, return decode(rqstp, &xdr, obj); } +static bool +gss_seq_is_newer(u32 new, u32 old) +{ + return (s32)(new - old) > 0; +} + +static bool +gss_xmit_need_reencode(struct rpc_task *task) +{ + struct rpc_rqst *req = task->tk_rqstp; + struct rpc_cred *cred = req->rq_cred; + struct gss_cl_ctx *ctx = gss_cred_get_ctx(cred); + u32 win, seq_xmit; + bool ret = true; + + if (!ctx) + return true; + + if (gss_seq_is_newer(req->rq_seqno, READ_ONCE(ctx->gc_seq))) + goto out; + + seq_xmit = READ_ONCE(ctx->gc_seq_xmit); + while (gss_seq_is_newer(req->rq_seqno, seq_xmit)) { + u32 tmp = seq_xmit; + + seq_xmit = cmpxchg(&ctx->gc_seq_xmit, tmp, req->rq_seqno); + if (seq_xmit == tmp) { + ret = false; + goto out; + } + } + + win = ctx->gc_win; + if (win > 0) + ret = !gss_seq_is_newer(req->rq_seqno, seq_xmit - win); +out: + gss_put_ctx(ctx); + return ret; +} + static int gss_unwrap_resp(struct rpc_task *task, kxdrdproc_t decode, void *rqstp, __be32 *p, void *obj) @@ -2052,6 +2092,7 @@ static const struct rpc_credops gss_credops = { .crunwrap_resp = gss_unwrap_resp, .crkey_timeout = gss_key_timeout, .crstringify_acceptor = gss_stringify_acceptor, + .crneed_reencode = gss_xmit_need_reencode, }; static const struct rpc_credops gss_nullops = { diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 4f1ec8013332..d41b5ac1d4e8 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2184,6 +2184,9 @@ call_status(struct rpc_task *task) /* shutdown or soft timeout */ rpc_exit(task, status); break; + case -EBADMSG: + task->tk_action = call_transmit; + break; default: if (clnt->cl_chatty) printk("%s: RPC call returned error %d\n", diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 6aa09edc9567..3973e10ea2bd 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1014,6 +1014,13 @@ void xprt_transmit(struct rpc_task *task) dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, req->rq_slen); if (!req->rq_reply_bytes_recvd) { + + /* Verify that our message lies in the RPCSEC_GSS window */ + if (!req->rq_bytes_sent && rpcauth_xmit_need_reencode(task)) { + task->tk_status = -EBADMSG; + return; + } + if (list_empty(&req->rq_list) && rpc_reply_expected(task)) { /* * Add to the list only if we're expecting a reply -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 04/34] SUNRPC: Simplify identification of when the message send/receive is complete 2018-09-04 21:05 ` [PATCH v2 03/34] SUNRPC: The transmitted message must lie in the RPCSEC window of validity Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 05/34] SUNRPC: Avoid holding locks across the XDR encoding of the RPC message Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Add states to indicate that the message send and receive are not yet complete. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/sched.h | 6 ++++-- net/sunrpc/clnt.c | 19 +++++++------------ net/sunrpc/xprt.c | 17 ++++++++++++++--- 3 files changed, 25 insertions(+), 17 deletions(-) diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index 592653becd91..9e655df70131 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -140,8 +140,10 @@ struct rpc_task_setup { #define RPC_TASK_RUNNING 0 #define RPC_TASK_QUEUED 1 #define RPC_TASK_ACTIVE 2 -#define RPC_TASK_MSG_RECV 3 -#define RPC_TASK_MSG_RECV_WAIT 4 +#define RPC_TASK_NEED_XMIT 3 +#define RPC_TASK_NEED_RECV 4 +#define RPC_TASK_MSG_RECV 5 +#define RPC_TASK_MSG_RECV_WAIT 6 #define RPC_IS_RUNNING(t) test_bit(RPC_TASK_RUNNING, &(t)->tk_runstate) #define rpc_set_running(t) set_bit(RPC_TASK_RUNNING, &(t)->tk_runstate) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index d41b5ac1d4e8..e5ac35e803ad 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1156,6 +1156,7 @@ struct rpc_task *rpc_run_bc_task(struct rpc_rqst *req) */ xbufp->len = xbufp->head[0].iov_len + xbufp->page_len + xbufp->tail[0].iov_len; + set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); task->tk_action = call_bc_transmit; atomic_inc(&task->tk_count); @@ -1720,17 +1721,10 @@ call_allocate(struct rpc_task *task) rpc_exit(task, -ERESTARTSYS); } -static inline int +static int rpc_task_need_encode(struct rpc_task *task) { - return task->tk_rqstp->rq_snd_buf.len == 0; -} - -static inline void -rpc_task_force_reencode(struct rpc_task *task) -{ - task->tk_rqstp->rq_snd_buf.len = 0; - task->tk_rqstp->rq_bytes_sent = 0; + return test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate) == 0; } /* @@ -1765,6 +1759,8 @@ rpc_xdr_encode(struct rpc_task *task) task->tk_status = rpcauth_wrap_req(task, encode, req, p, task->tk_msg.rpc_argp); + if (task->tk_status == 0) + set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); } /* @@ -1999,7 +1995,6 @@ call_transmit_status(struct rpc_task *task) */ if (task->tk_status == 0) { xprt_end_transmit(task); - rpc_task_force_reencode(task); return; } @@ -2010,7 +2005,6 @@ call_transmit_status(struct rpc_task *task) default: dprint_status(task); xprt_end_transmit(task); - rpc_task_force_reencode(task); break; /* * Special cases: if we've been waiting on the @@ -2038,7 +2032,7 @@ call_transmit_status(struct rpc_task *task) case -EADDRINUSE: case -ENOTCONN: case -EPIPE: - rpc_task_force_reencode(task); + break; } } @@ -2185,6 +2179,7 @@ call_status(struct rpc_task *task) rpc_exit(task, status); break; case -EBADMSG: + clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); task->tk_action = call_transmit; break; default: diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 3973e10ea2bd..45d580cd93ac 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -936,10 +936,18 @@ void xprt_complete_rqst(struct rpc_task *task, int copied) /* req->rq_reply_bytes_recvd */ smp_wmb(); req->rq_reply_bytes_recvd = copied; + clear_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); rpc_wake_up_queued_task(&xprt->pending, task); } EXPORT_SYMBOL_GPL(xprt_complete_rqst); +static bool +xprt_request_data_received(struct rpc_task *task) +{ + return !test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate) && + task->tk_rqstp->rq_reply_bytes_recvd != 0; +} + static void xprt_timer(struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; @@ -1031,12 +1039,13 @@ void xprt_transmit(struct rpc_task *task) /* Add request to the receive list */ spin_lock(&xprt->recv_lock); list_add_tail(&req->rq_list, &xprt->recv); + set_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); spin_unlock(&xprt->recv_lock); xprt_reset_majortimeo(req); /* Turn off autodisconnect */ del_singleshot_timer_sync(&xprt->timer); } - } else if (!req->rq_bytes_sent) + } else if (xprt_request_data_received(task) && !req->rq_bytes_sent) return; connect_cookie = xprt->connect_cookie; @@ -1046,9 +1055,11 @@ void xprt_transmit(struct rpc_task *task) task->tk_status = status; return; } + xprt_inject_disconnect(xprt); dprintk("RPC: %5u xmit complete\n", task->tk_pid); + clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); task->tk_flags |= RPC_TASK_SENT; spin_lock_bh(&xprt->transport_lock); @@ -1062,14 +1073,14 @@ void xprt_transmit(struct rpc_task *task) spin_unlock_bh(&xprt->transport_lock); req->rq_connect_cookie = connect_cookie; - if (rpc_reply_expected(task) && !READ_ONCE(req->rq_reply_bytes_recvd)) { + if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { /* * Sleep on the pending queue if we're expecting a reply. * The spinlock ensures atomicity between the test of * req->rq_reply_bytes_recvd, and the call to rpc_sleep_on(). */ spin_lock(&xprt->recv_lock); - if (!req->rq_reply_bytes_recvd) { + if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { rpc_sleep_on(&xprt->pending, task, xprt_timer); /* * Send an extra queue wakeup call if the -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 05/34] SUNRPC: Avoid holding locks across the XDR encoding of the RPC message 2018-09-04 21:05 ` [PATCH v2 04/34] SUNRPC: Simplify identification of when the message send/receive is complete Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 06/34] SUNRPC: Rename TCP receive-specific state variables Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Currently, we grab the socket bit lock before we allow the message to be XDR encoded. That significantly slows down the transmission rate, since we serialise on a potentially blocking operation. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index e5ac35e803ad..a858366cd15d 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1949,9 +1949,6 @@ call_transmit(struct rpc_task *task) task->tk_action = call_status; if (task->tk_status < 0) return; - if (!xprt_prepare_transmit(task)) - return; - task->tk_action = call_transmit_status; /* Encode here so that rpcsec_gss can use correct sequence number. */ if (rpc_task_need_encode(task)) { rpc_xdr_encode(task); @@ -1965,6 +1962,9 @@ call_transmit(struct rpc_task *task) return; } } + if (!xprt_prepare_transmit(task)) + return; + task->tk_action = call_transmit_status; xprt_transmit(task); if (task->tk_status < 0) return; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 06/34] SUNRPC: Rename TCP receive-specific state variables 2018-09-04 21:05 ` [PATCH v2 05/34] SUNRPC: Avoid holding locks across the XDR encoding of the RPC message Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 07/34] SUNRPC: Move reset of TCP state variables into the reconnect code Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Since we will want to introduce similar TCP state variables for the transmission of requests, let's rename the existing ones to label that they are for the receive side. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprtsock.h | 16 +-- include/trace/events/sunrpc.h | 10 +- net/sunrpc/xprtsock.c | 178 ++++++++++++++++---------------- 3 files changed, 103 insertions(+), 101 deletions(-) diff --git a/include/linux/sunrpc/xprtsock.h b/include/linux/sunrpc/xprtsock.h index ae0f99b9b965..90d5ca8e65f4 100644 --- a/include/linux/sunrpc/xprtsock.h +++ b/include/linux/sunrpc/xprtsock.h @@ -30,15 +30,17 @@ struct sock_xprt { /* * State of TCP reply receive */ - __be32 tcp_fraghdr, - tcp_xid, - tcp_calldir; + struct { + __be32 fraghdr, + xid, + calldir; - u32 tcp_offset, - tcp_reclen; + u32 offset, + len; - unsigned long tcp_copied, - tcp_flags; + unsigned long copied, + flags; + } recv; /* * Connection of transports diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index bbb08a3ef5cc..0aa347194e0f 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -525,11 +525,11 @@ TRACE_EVENT(xs_tcp_data_recv, TP_fast_assign( __assign_str(addr, xs->xprt.address_strings[RPC_DISPLAY_ADDR]); __assign_str(port, xs->xprt.address_strings[RPC_DISPLAY_PORT]); - __entry->xid = be32_to_cpu(xs->tcp_xid); - __entry->flags = xs->tcp_flags; - __entry->copied = xs->tcp_copied; - __entry->reclen = xs->tcp_reclen; - __entry->offset = xs->tcp_offset; + __entry->xid = be32_to_cpu(xs->recv.xid); + __entry->flags = xs->recv.flags; + __entry->copied = xs->recv.copied; + __entry->reclen = xs->recv.len; + __entry->offset = xs->recv.offset; ), TP_printk("peer=[%s]:%s xid=0x%08x flags=%s copied=%lu reclen=%u offset=%lu", diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 6b7539c0466e..cd7d093721ae 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -1169,42 +1169,42 @@ static inline void xs_tcp_read_fraghdr(struct rpc_xprt *xprt, struct xdr_skb_rea size_t len, used; char *p; - p = ((char *) &transport->tcp_fraghdr) + transport->tcp_offset; - len = sizeof(transport->tcp_fraghdr) - transport->tcp_offset; + p = ((char *) &transport->recv.fraghdr) + transport->recv.offset; + len = sizeof(transport->recv.fraghdr) - transport->recv.offset; used = xdr_skb_read_bits(desc, p, len); - transport->tcp_offset += used; + transport->recv.offset += used; if (used != len) return; - transport->tcp_reclen = ntohl(transport->tcp_fraghdr); - if (transport->tcp_reclen & RPC_LAST_STREAM_FRAGMENT) - transport->tcp_flags |= TCP_RCV_LAST_FRAG; + transport->recv.len = ntohl(transport->recv.fraghdr); + if (transport->recv.len & RPC_LAST_STREAM_FRAGMENT) + transport->recv.flags |= TCP_RCV_LAST_FRAG; else - transport->tcp_flags &= ~TCP_RCV_LAST_FRAG; - transport->tcp_reclen &= RPC_FRAGMENT_SIZE_MASK; + transport->recv.flags &= ~TCP_RCV_LAST_FRAG; + transport->recv.len &= RPC_FRAGMENT_SIZE_MASK; - transport->tcp_flags &= ~TCP_RCV_COPY_FRAGHDR; - transport->tcp_offset = 0; + transport->recv.flags &= ~TCP_RCV_COPY_FRAGHDR; + transport->recv.offset = 0; /* Sanity check of the record length */ - if (unlikely(transport->tcp_reclen < 8)) { + if (unlikely(transport->recv.len < 8)) { dprintk("RPC: invalid TCP record fragment length\n"); xs_tcp_force_close(xprt); return; } dprintk("RPC: reading TCP record fragment of length %d\n", - transport->tcp_reclen); + transport->recv.len); } static void xs_tcp_check_fraghdr(struct sock_xprt *transport) { - if (transport->tcp_offset == transport->tcp_reclen) { - transport->tcp_flags |= TCP_RCV_COPY_FRAGHDR; - transport->tcp_offset = 0; - if (transport->tcp_flags & TCP_RCV_LAST_FRAG) { - transport->tcp_flags &= ~TCP_RCV_COPY_DATA; - transport->tcp_flags |= TCP_RCV_COPY_XID; - transport->tcp_copied = 0; + if (transport->recv.offset == transport->recv.len) { + transport->recv.flags |= TCP_RCV_COPY_FRAGHDR; + transport->recv.offset = 0; + if (transport->recv.flags & TCP_RCV_LAST_FRAG) { + transport->recv.flags &= ~TCP_RCV_COPY_DATA; + transport->recv.flags |= TCP_RCV_COPY_XID; + transport->recv.copied = 0; } } } @@ -1214,20 +1214,20 @@ static inline void xs_tcp_read_xid(struct sock_xprt *transport, struct xdr_skb_r size_t len, used; char *p; - len = sizeof(transport->tcp_xid) - transport->tcp_offset; + len = sizeof(transport->recv.xid) - transport->recv.offset; dprintk("RPC: reading XID (%zu bytes)\n", len); - p = ((char *) &transport->tcp_xid) + transport->tcp_offset; + p = ((char *) &transport->recv.xid) + transport->recv.offset; used = xdr_skb_read_bits(desc, p, len); - transport->tcp_offset += used; + transport->recv.offset += used; if (used != len) return; - transport->tcp_flags &= ~TCP_RCV_COPY_XID; - transport->tcp_flags |= TCP_RCV_READ_CALLDIR; - transport->tcp_copied = 4; + transport->recv.flags &= ~TCP_RCV_COPY_XID; + transport->recv.flags |= TCP_RCV_READ_CALLDIR; + transport->recv.copied = 4; dprintk("RPC: reading %s XID %08x\n", - (transport->tcp_flags & TCP_RPC_REPLY) ? "reply for" + (transport->recv.flags & TCP_RPC_REPLY) ? "reply for" : "request with", - ntohl(transport->tcp_xid)); + ntohl(transport->recv.xid)); xs_tcp_check_fraghdr(transport); } @@ -1239,34 +1239,34 @@ static inline void xs_tcp_read_calldir(struct sock_xprt *transport, char *p; /* - * We want transport->tcp_offset to be 8 at the end of this routine + * We want transport->recv.offset to be 8 at the end of this routine * (4 bytes for the xid and 4 bytes for the call/reply flag). * When this function is called for the first time, - * transport->tcp_offset is 4 (after having already read the xid). + * transport->recv.offset is 4 (after having already read the xid). */ - offset = transport->tcp_offset - sizeof(transport->tcp_xid); - len = sizeof(transport->tcp_calldir) - offset; + offset = transport->recv.offset - sizeof(transport->recv.xid); + len = sizeof(transport->recv.calldir) - offset; dprintk("RPC: reading CALL/REPLY flag (%zu bytes)\n", len); - p = ((char *) &transport->tcp_calldir) + offset; + p = ((char *) &transport->recv.calldir) + offset; used = xdr_skb_read_bits(desc, p, len); - transport->tcp_offset += used; + transport->recv.offset += used; if (used != len) return; - transport->tcp_flags &= ~TCP_RCV_READ_CALLDIR; + transport->recv.flags &= ~TCP_RCV_READ_CALLDIR; /* * We don't yet have the XDR buffer, so we will write the calldir * out after we get the buffer from the 'struct rpc_rqst' */ - switch (ntohl(transport->tcp_calldir)) { + switch (ntohl(transport->recv.calldir)) { case RPC_REPLY: - transport->tcp_flags |= TCP_RCV_COPY_CALLDIR; - transport->tcp_flags |= TCP_RCV_COPY_DATA; - transport->tcp_flags |= TCP_RPC_REPLY; + transport->recv.flags |= TCP_RCV_COPY_CALLDIR; + transport->recv.flags |= TCP_RCV_COPY_DATA; + transport->recv.flags |= TCP_RPC_REPLY; break; case RPC_CALL: - transport->tcp_flags |= TCP_RCV_COPY_CALLDIR; - transport->tcp_flags |= TCP_RCV_COPY_DATA; - transport->tcp_flags &= ~TCP_RPC_REPLY; + transport->recv.flags |= TCP_RCV_COPY_CALLDIR; + transport->recv.flags |= TCP_RCV_COPY_DATA; + transport->recv.flags &= ~TCP_RPC_REPLY; break; default: dprintk("RPC: invalid request message type\n"); @@ -1287,21 +1287,21 @@ static inline void xs_tcp_read_common(struct rpc_xprt *xprt, rcvbuf = &req->rq_private_buf; - if (transport->tcp_flags & TCP_RCV_COPY_CALLDIR) { + if (transport->recv.flags & TCP_RCV_COPY_CALLDIR) { /* * Save the RPC direction in the XDR buffer */ - memcpy(rcvbuf->head[0].iov_base + transport->tcp_copied, - &transport->tcp_calldir, - sizeof(transport->tcp_calldir)); - transport->tcp_copied += sizeof(transport->tcp_calldir); - transport->tcp_flags &= ~TCP_RCV_COPY_CALLDIR; + memcpy(rcvbuf->head[0].iov_base + transport->recv.copied, + &transport->recv.calldir, + sizeof(transport->recv.calldir)); + transport->recv.copied += sizeof(transport->recv.calldir); + transport->recv.flags &= ~TCP_RCV_COPY_CALLDIR; } len = desc->count; - if (len > transport->tcp_reclen - transport->tcp_offset) - desc->count = transport->tcp_reclen - transport->tcp_offset; - r = xdr_partial_copy_from_skb(rcvbuf, transport->tcp_copied, + if (len > transport->recv.len - transport->recv.offset) + desc->count = transport->recv.len - transport->recv.offset; + r = xdr_partial_copy_from_skb(rcvbuf, transport->recv.copied, desc, xdr_skb_read_bits); if (desc->count) { @@ -1314,31 +1314,31 @@ static inline void xs_tcp_read_common(struct rpc_xprt *xprt, * Any remaining data from this record will * be discarded. */ - transport->tcp_flags &= ~TCP_RCV_COPY_DATA; + transport->recv.flags &= ~TCP_RCV_COPY_DATA; dprintk("RPC: XID %08x truncated request\n", - ntohl(transport->tcp_xid)); - dprintk("RPC: xprt = %p, tcp_copied = %lu, " - "tcp_offset = %u, tcp_reclen = %u\n", - xprt, transport->tcp_copied, - transport->tcp_offset, transport->tcp_reclen); + ntohl(transport->recv.xid)); + dprintk("RPC: xprt = %p, recv.copied = %lu, " + "recv.offset = %u, recv.len = %u\n", + xprt, transport->recv.copied, + transport->recv.offset, transport->recv.len); return; } - transport->tcp_copied += r; - transport->tcp_offset += r; + transport->recv.copied += r; + transport->recv.offset += r; desc->count = len - r; dprintk("RPC: XID %08x read %zd bytes\n", - ntohl(transport->tcp_xid), r); - dprintk("RPC: xprt = %p, tcp_copied = %lu, tcp_offset = %u, " - "tcp_reclen = %u\n", xprt, transport->tcp_copied, - transport->tcp_offset, transport->tcp_reclen); - - if (transport->tcp_copied == req->rq_private_buf.buflen) - transport->tcp_flags &= ~TCP_RCV_COPY_DATA; - else if (transport->tcp_offset == transport->tcp_reclen) { - if (transport->tcp_flags & TCP_RCV_LAST_FRAG) - transport->tcp_flags &= ~TCP_RCV_COPY_DATA; + ntohl(transport->recv.xid), r); + dprintk("RPC: xprt = %p, recv.copied = %lu, recv.offset = %u, " + "recv.len = %u\n", xprt, transport->recv.copied, + transport->recv.offset, transport->recv.len); + + if (transport->recv.copied == req->rq_private_buf.buflen) + transport->recv.flags &= ~TCP_RCV_COPY_DATA; + else if (transport->recv.offset == transport->recv.len) { + if (transport->recv.flags & TCP_RCV_LAST_FRAG) + transport->recv.flags &= ~TCP_RCV_COPY_DATA; } } @@ -1353,14 +1353,14 @@ static inline int xs_tcp_read_reply(struct rpc_xprt *xprt, container_of(xprt, struct sock_xprt, xprt); struct rpc_rqst *req; - dprintk("RPC: read reply XID %08x\n", ntohl(transport->tcp_xid)); + dprintk("RPC: read reply XID %08x\n", ntohl(transport->recv.xid)); /* Find and lock the request corresponding to this xid */ spin_lock(&xprt->recv_lock); - req = xprt_lookup_rqst(xprt, transport->tcp_xid); + req = xprt_lookup_rqst(xprt, transport->recv.xid); if (!req) { dprintk("RPC: XID %08x request not found!\n", - ntohl(transport->tcp_xid)); + ntohl(transport->recv.xid)); spin_unlock(&xprt->recv_lock); return -1; } @@ -1370,8 +1370,8 @@ static inline int xs_tcp_read_reply(struct rpc_xprt *xprt, xs_tcp_read_common(xprt, desc, req); spin_lock(&xprt->recv_lock); - if (!(transport->tcp_flags & TCP_RCV_COPY_DATA)) - xprt_complete_rqst(req->rq_task, transport->tcp_copied); + if (!(transport->recv.flags & TCP_RCV_COPY_DATA)) + xprt_complete_rqst(req->rq_task, transport->recv.copied); xprt_unpin_rqst(req); spin_unlock(&xprt->recv_lock); return 0; @@ -1393,7 +1393,7 @@ static int xs_tcp_read_callback(struct rpc_xprt *xprt, struct rpc_rqst *req; /* Look up the request corresponding to the given XID */ - req = xprt_lookup_bc_request(xprt, transport->tcp_xid); + req = xprt_lookup_bc_request(xprt, transport->recv.xid); if (req == NULL) { printk(KERN_WARNING "Callback slot table overflowed\n"); xprt_force_disconnect(xprt); @@ -1403,8 +1403,8 @@ static int xs_tcp_read_callback(struct rpc_xprt *xprt, dprintk("RPC: read callback XID %08x\n", ntohl(req->rq_xid)); xs_tcp_read_common(xprt, desc, req); - if (!(transport->tcp_flags & TCP_RCV_COPY_DATA)) - xprt_complete_bc_request(req, transport->tcp_copied); + if (!(transport->recv.flags & TCP_RCV_COPY_DATA)) + xprt_complete_bc_request(req, transport->recv.copied); return 0; } @@ -1415,7 +1415,7 @@ static inline int _xs_tcp_read_data(struct rpc_xprt *xprt, struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); - return (transport->tcp_flags & TCP_RPC_REPLY) ? + return (transport->recv.flags & TCP_RPC_REPLY) ? xs_tcp_read_reply(xprt, desc) : xs_tcp_read_callback(xprt, desc); } @@ -1458,9 +1458,9 @@ static void xs_tcp_read_data(struct rpc_xprt *xprt, else { /* * The transport_lock protects the request handling. - * There's no need to hold it to update the tcp_flags. + * There's no need to hold it to update the recv.flags. */ - transport->tcp_flags &= ~TCP_RCV_COPY_DATA; + transport->recv.flags &= ~TCP_RCV_COPY_DATA; } } @@ -1468,12 +1468,12 @@ static inline void xs_tcp_read_discard(struct sock_xprt *transport, struct xdr_s { size_t len; - len = transport->tcp_reclen - transport->tcp_offset; + len = transport->recv.len - transport->recv.offset; if (len > desc->count) len = desc->count; desc->count -= len; desc->offset += len; - transport->tcp_offset += len; + transport->recv.offset += len; dprintk("RPC: discarded %zu bytes\n", len); xs_tcp_check_fraghdr(transport); } @@ -1494,22 +1494,22 @@ static int xs_tcp_data_recv(read_descriptor_t *rd_desc, struct sk_buff *skb, uns trace_xs_tcp_data_recv(transport); /* Read in a new fragment marker if necessary */ /* Can we ever really expect to get completely empty fragments? */ - if (transport->tcp_flags & TCP_RCV_COPY_FRAGHDR) { + if (transport->recv.flags & TCP_RCV_COPY_FRAGHDR) { xs_tcp_read_fraghdr(xprt, &desc); continue; } /* Read in the xid if necessary */ - if (transport->tcp_flags & TCP_RCV_COPY_XID) { + if (transport->recv.flags & TCP_RCV_COPY_XID) { xs_tcp_read_xid(transport, &desc); continue; } /* Read in the call/reply flag */ - if (transport->tcp_flags & TCP_RCV_READ_CALLDIR) { + if (transport->recv.flags & TCP_RCV_READ_CALLDIR) { xs_tcp_read_calldir(transport, &desc); continue; } /* Read in the request data */ - if (transport->tcp_flags & TCP_RCV_COPY_DATA) { + if (transport->recv.flags & TCP_RCV_COPY_DATA) { xs_tcp_read_data(xprt, &desc); continue; } @@ -1602,10 +1602,10 @@ static void xs_tcp_state_change(struct sock *sk) if (!xprt_test_and_set_connected(xprt)) { /* Reset TCP record info */ - transport->tcp_offset = 0; - transport->tcp_reclen = 0; - transport->tcp_copied = 0; - transport->tcp_flags = + transport->recv.offset = 0; + transport->recv.len = 0; + transport->recv.copied = 0; + transport->recv.flags = TCP_RCV_COPY_FRAGHDR | TCP_RCV_COPY_XID; xprt->connect_cookie++; clear_bit(XPRT_SOCK_CONNECTING, &transport->sock_state); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 07/34] SUNRPC: Move reset of TCP state variables into the reconnect code 2018-09-04 21:05 ` [PATCH v2 06/34] SUNRPC: Rename TCP receive-specific state variables Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 08/34] SUNRPC: Add socket transmit queue offset tracking Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Rather than resetting state variables in socket state_change() callback, do it in the sunrpc TCP connect function itself. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprtsock.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index cd7d093721ae..ec1e3f93e707 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -1600,13 +1600,6 @@ static void xs_tcp_state_change(struct sock *sk) case TCP_ESTABLISHED: spin_lock(&xprt->transport_lock); if (!xprt_test_and_set_connected(xprt)) { - - /* Reset TCP record info */ - transport->recv.offset = 0; - transport->recv.len = 0; - transport->recv.copied = 0; - transport->recv.flags = - TCP_RCV_COPY_FRAGHDR | TCP_RCV_COPY_XID; xprt->connect_cookie++; clear_bit(XPRT_SOCK_CONNECTING, &transport->sock_state); xprt_clear_connecting(xprt); @@ -2386,6 +2379,12 @@ static int xs_tcp_finish_connecting(struct rpc_xprt *xprt, struct socket *sock) xs_set_memalloc(xprt); + /* Reset TCP record info */ + transport->recv.offset = 0; + transport->recv.len = 0; + transport->recv.copied = 0; + transport->recv.flags = TCP_RCV_COPY_FRAGHDR | TCP_RCV_COPY_XID; + /* Tell the socket layer to start connecting... */ xprt->stat.connect_count++; xprt->stat.connect_start = jiffies; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 08/34] SUNRPC: Add socket transmit queue offset tracking 2018-09-04 21:05 ` [PATCH v2 07/34] SUNRPC: Move reset of TCP state variables into the reconnect code Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 09/34] SUNRPC: Simplify dealing with aborted partially transmitted messages Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprtsock.h | 7 ++++++ net/sunrpc/xprtsock.c | 40 ++++++++++++++++++--------------- 2 files changed, 29 insertions(+), 18 deletions(-) diff --git a/include/linux/sunrpc/xprtsock.h b/include/linux/sunrpc/xprtsock.h index 90d5ca8e65f4..005cfb6e7238 100644 --- a/include/linux/sunrpc/xprtsock.h +++ b/include/linux/sunrpc/xprtsock.h @@ -42,6 +42,13 @@ struct sock_xprt { flags; } recv; + /* + * State of TCP transmit queue + */ + struct { + u32 offset; + } xmit; + /* * Connection of transports */ diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index ec1e3f93e707..629cc45e1e6c 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -461,7 +461,7 @@ static int xs_nospace(struct rpc_task *task) int ret = -EAGAIN; dprintk("RPC: %5u xmit incomplete (%u left of %u)\n", - task->tk_pid, req->rq_slen - req->rq_bytes_sent, + task->tk_pid, req->rq_slen - transport->xmit.offset, req->rq_slen); /* Protect against races with write_space */ @@ -528,19 +528,22 @@ static int xs_local_send_request(struct rpc_task *task) req->rq_svec->iov_base, req->rq_svec->iov_len); req->rq_xtime = ktime_get(); - status = xs_sendpages(transport->sock, NULL, 0, xdr, req->rq_bytes_sent, + status = xs_sendpages(transport->sock, NULL, 0, xdr, + transport->xmit.offset, true, &sent); dprintk("RPC: %s(%u) = %d\n", - __func__, xdr->len - req->rq_bytes_sent, status); + __func__, xdr->len - transport->xmit.offset, status); if (status == -EAGAIN && sock_writeable(transport->inet)) status = -ENOBUFS; if (likely(sent > 0) || status == 0) { - req->rq_bytes_sent += sent; - req->rq_xmit_bytes_sent += sent; + transport->xmit.offset += sent; + req->rq_bytes_sent = transport->xmit.offset; if (likely(req->rq_bytes_sent >= req->rq_slen)) { + req->rq_xmit_bytes_sent += transport->xmit.offset; req->rq_bytes_sent = 0; + transport->xmit.offset = 0; return 0; } status = -EAGAIN; @@ -592,10 +595,10 @@ static int xs_udp_send_request(struct rpc_task *task) return -ENOTCONN; req->rq_xtime = ktime_get(); status = xs_sendpages(transport->sock, xs_addr(xprt), xprt->addrlen, - xdr, req->rq_bytes_sent, true, &sent); + xdr, 0, true, &sent); dprintk("RPC: xs_udp_send_request(%u) = %d\n", - xdr->len - req->rq_bytes_sent, status); + xdr->len, status); /* firewall is blocking us, don't return -EAGAIN or we end up looping */ if (status == -EPERM) @@ -684,17 +687,20 @@ static int xs_tcp_send_request(struct rpc_task *task) while (1) { sent = 0; status = xs_sendpages(transport->sock, NULL, 0, xdr, - req->rq_bytes_sent, zerocopy, &sent); + transport->xmit.offset, + zerocopy, &sent); dprintk("RPC: xs_tcp_send_request(%u) = %d\n", - xdr->len - req->rq_bytes_sent, status); + xdr->len - transport->xmit.offset, status); /* If we've sent the entire packet, immediately * reset the count of bytes sent. */ - req->rq_bytes_sent += sent; - req->rq_xmit_bytes_sent += sent; + transport->xmit.offset += sent; + req->rq_bytes_sent = transport->xmit.offset; if (likely(req->rq_bytes_sent >= req->rq_slen)) { + req->rq_xmit_bytes_sent += transport->xmit.offset; req->rq_bytes_sent = 0; + transport->xmit.offset = 0; return 0; } @@ -760,18 +766,13 @@ static int xs_tcp_send_request(struct rpc_task *task) */ static void xs_tcp_release_xprt(struct rpc_xprt *xprt, struct rpc_task *task) { - struct rpc_rqst *req; + struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); if (task != xprt->snd_task) return; if (task == NULL) goto out_release; - req = task->tk_rqstp; - if (req == NULL) - goto out_release; - if (req->rq_bytes_sent == 0) - goto out_release; - if (req->rq_bytes_sent == req->rq_snd_buf.len) + if (transport->xmit.offset == 0 || !xprt_connected(xprt)) goto out_release; set_bit(XPRT_CLOSE_WAIT, &xprt->state); out_release: @@ -2021,6 +2022,8 @@ static int xs_local_finish_connecting(struct rpc_xprt *xprt, write_unlock_bh(&sk->sk_callback_lock); } + transport->xmit.offset = 0; + /* Tell the socket layer to start connecting... */ xprt->stat.connect_count++; xprt->stat.connect_start = jiffies; @@ -2384,6 +2387,7 @@ static int xs_tcp_finish_connecting(struct rpc_xprt *xprt, struct socket *sock) transport->recv.len = 0; transport->recv.copied = 0; transport->recv.flags = TCP_RCV_COPY_FRAGHDR | TCP_RCV_COPY_XID; + transport->xmit.offset = 0; /* Tell the socket layer to start connecting... */ xprt->stat.connect_count++; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 09/34] SUNRPC: Simplify dealing with aborted partially transmitted messages 2018-09-04 21:05 ` [PATCH v2 08/34] SUNRPC: Add socket transmit queue offset tracking Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 10/34] SUNRPC: Refactor the transport request pinning Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs If the previous message was only partially transmitted, we need to close the socket in order to avoid corruption of the message stream. To do so, we currently hijack the unlocking of the socket in order to schedule the close. Now that we track the message offset in the socket state, we can move that kind of checking out of the socket lock code, which is needed to allow messages to remain queued after dropping the socket lock. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprtsock.c | 51 +++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 629cc45e1e6c..3fbccebd0b10 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -491,6 +491,16 @@ static int xs_nospace(struct rpc_task *task) return ret; } +/* + * Determine if the previous message in the stream was aborted before it + * could complete transmission. + */ +static bool +xs_send_request_was_aborted(struct sock_xprt *transport, struct rpc_rqst *req) +{ + return transport->xmit.offset != 0 && req->rq_bytes_sent == 0; +} + /* * Construct a stream transport record marker in @buf. */ @@ -522,6 +532,12 @@ static int xs_local_send_request(struct rpc_task *task) int status; int sent = 0; + /* Close the stream if the previous transmission was incomplete */ + if (xs_send_request_was_aborted(transport, req)) { + xs_close(xprt); + return -ENOTCONN; + } + xs_encode_stream_record_marker(&req->rq_snd_buf); xs_pktdump("packet data:", @@ -665,6 +681,13 @@ static int xs_tcp_send_request(struct rpc_task *task) int status; int sent; + /* Close the stream if the previous transmission was incomplete */ + if (xs_send_request_was_aborted(transport, req)) { + if (transport->sock != NULL) + kernel_sock_shutdown(transport->sock, SHUT_RDWR); + return -ENOTCONN; + } + xs_encode_stream_record_marker(&req->rq_snd_buf); xs_pktdump("packet data:", @@ -755,30 +778,6 @@ static int xs_tcp_send_request(struct rpc_task *task) return status; } -/** - * xs_tcp_release_xprt - clean up after a tcp transmission - * @xprt: transport - * @task: rpc task - * - * This cleans up if an error causes us to abort the transmission of a request. - * In this case, the socket may need to be reset in order to avoid confusing - * the server. - */ -static void xs_tcp_release_xprt(struct rpc_xprt *xprt, struct rpc_task *task) -{ - struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); - - if (task != xprt->snd_task) - return; - if (task == NULL) - goto out_release; - if (transport->xmit.offset == 0 || !xprt_connected(xprt)) - goto out_release; - set_bit(XPRT_CLOSE_WAIT, &xprt->state); -out_release: - xprt_release_xprt(xprt, task); -} - static void xs_save_old_callbacks(struct sock_xprt *transport, struct sock *sk) { transport->old_data_ready = sk->sk_data_ready; @@ -2764,7 +2763,7 @@ static void bc_destroy(struct rpc_xprt *xprt) static const struct rpc_xprt_ops xs_local_ops = { .reserve_xprt = xprt_reserve_xprt, - .release_xprt = xs_tcp_release_xprt, + .release_xprt = xprt_release_xprt, .alloc_slot = xprt_alloc_slot, .free_slot = xprt_free_slot, .rpcbind = xs_local_rpcbind, @@ -2806,7 +2805,7 @@ static const struct rpc_xprt_ops xs_udp_ops = { static const struct rpc_xprt_ops xs_tcp_ops = { .reserve_xprt = xprt_reserve_xprt, - .release_xprt = xs_tcp_release_xprt, + .release_xprt = xprt_release_xprt, .alloc_slot = xprt_lock_and_alloc_slot, .free_slot = xprt_free_slot, .rpcbind = rpcb_getport_async, -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 10/34] SUNRPC: Refactor the transport request pinning 2018-09-04 21:05 ` [PATCH v2 09/34] SUNRPC: Simplify dealing with aborted partially transmitted messages Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 11/34] SUNRPC: Add a helper to wake up a sleeping rpc_task and set its status Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs We are going to need to pin for both send and receive. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/sched.h | 2 -- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/xprt.c | 37 +++++++++++++++++------------------- 3 files changed, 18 insertions(+), 22 deletions(-) diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index 9e655df70131..a4a42b3a1f03 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -142,8 +142,6 @@ struct rpc_task_setup { #define RPC_TASK_ACTIVE 2 #define RPC_TASK_NEED_XMIT 3 #define RPC_TASK_NEED_RECV 4 -#define RPC_TASK_MSG_RECV 5 -#define RPC_TASK_MSG_RECV_WAIT 6 #define RPC_IS_RUNNING(t) test_bit(RPC_TASK_RUNNING, &(t)->tk_runstate) #define rpc_set_running(t) set_bit(RPC_TASK_RUNNING, &(t)->tk_runstate) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 3d80524e92d6..bd743c51a865 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -103,6 +103,7 @@ struct rpc_rqst { /* A cookie used to track the state of the transport connection */ + atomic_t rq_pin; /* * Partial send handling diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 45d580cd93ac..bf8fc1a5dbd1 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -847,16 +847,22 @@ struct rpc_rqst *xprt_lookup_rqst(struct rpc_xprt *xprt, __be32 xid) } EXPORT_SYMBOL_GPL(xprt_lookup_rqst); +static bool +xprt_is_pinned_rqst(struct rpc_rqst *req) +{ + return atomic_read(&req->rq_pin) != 0; +} + /** * xprt_pin_rqst - Pin a request on the transport receive list * @req: Request to pin * * Caller must ensure this is atomic with the call to xprt_lookup_rqst() - * so should be holding the xprt transport lock. + * so should be holding the xprt receive lock. */ void xprt_pin_rqst(struct rpc_rqst *req) { - set_bit(RPC_TASK_MSG_RECV, &req->rq_task->tk_runstate); + atomic_inc(&req->rq_pin); } EXPORT_SYMBOL_GPL(xprt_pin_rqst); @@ -864,31 +870,18 @@ EXPORT_SYMBOL_GPL(xprt_pin_rqst); * xprt_unpin_rqst - Unpin a request on the transport receive list * @req: Request to pin * - * Caller should be holding the xprt transport lock. + * Caller should be holding the xprt receive lock. */ void xprt_unpin_rqst(struct rpc_rqst *req) { - struct rpc_task *task = req->rq_task; - - clear_bit(RPC_TASK_MSG_RECV, &task->tk_runstate); - if (test_bit(RPC_TASK_MSG_RECV_WAIT, &task->tk_runstate)) - wake_up_bit(&task->tk_runstate, RPC_TASK_MSG_RECV); + if (atomic_dec_and_test(&req->rq_pin)) + wake_up_var(&req->rq_pin); } EXPORT_SYMBOL_GPL(xprt_unpin_rqst); static void xprt_wait_on_pinned_rqst(struct rpc_rqst *req) -__must_hold(&req->rq_xprt->recv_lock) { - struct rpc_task *task = req->rq_task; - - if (task && test_bit(RPC_TASK_MSG_RECV, &task->tk_runstate)) { - spin_unlock(&req->rq_xprt->recv_lock); - set_bit(RPC_TASK_MSG_RECV_WAIT, &task->tk_runstate); - wait_on_bit(&task->tk_runstate, RPC_TASK_MSG_RECV, - TASK_UNINTERRUPTIBLE); - clear_bit(RPC_TASK_MSG_RECV_WAIT, &task->tk_runstate); - spin_lock(&req->rq_xprt->recv_lock); - } + wait_var_event(&req->rq_pin, !xprt_is_pinned_rqst(req)); } /** @@ -1388,7 +1381,11 @@ void xprt_release(struct rpc_task *task) spin_lock(&xprt->recv_lock); if (!list_empty(&req->rq_list)) { list_del_init(&req->rq_list); - xprt_wait_on_pinned_rqst(req); + if (atomic_read(&req->rq_pin)) { + spin_unlock(&xprt->recv_lock); + xprt_wait_on_pinned_rqst(req); + spin_lock(&xprt->recv_lock); + } } spin_unlock(&xprt->recv_lock); spin_lock_bh(&xprt->transport_lock); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 11/34] SUNRPC: Add a helper to wake up a sleeping rpc_task and set its status 2018-09-04 21:05 ` [PATCH v2 10/34] SUNRPC: Refactor the transport request pinning Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 12/34] SUNRPC: Don't wake queued RPC calls multiple times in xprt_transmit Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Add a helper that will wake up a task that is sleeping on a specific queue, and will set the value of task->tk_status. This is mainly intended for use by the transport layer to notify the task of an error condition. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/sched.h | 3 ++ net/sunrpc/sched.c | 63 ++++++++++++++++++++++++++++++------ 2 files changed, 56 insertions(+), 10 deletions(-) diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index a4a42b3a1f03..c5bc779feb00 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -234,6 +234,9 @@ void rpc_wake_up_queued_task_on_wq(struct workqueue_struct *wq, struct rpc_task *task); void rpc_wake_up_queued_task(struct rpc_wait_queue *, struct rpc_task *); +void rpc_wake_up_queued_task_set_status(struct rpc_wait_queue *, + struct rpc_task *, + int); void rpc_wake_up(struct rpc_wait_queue *); struct rpc_task *rpc_wake_up_next(struct rpc_wait_queue *); struct rpc_task *rpc_wake_up_first_on_wq(struct workqueue_struct *wq, diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c index 3fe5d60ab0e2..104c056daf83 100644 --- a/net/sunrpc/sched.c +++ b/net/sunrpc/sched.c @@ -440,14 +440,28 @@ static void __rpc_do_wake_up_task_on_wq(struct workqueue_struct *wq, /* * Wake up a queued task while the queue lock is being held */ -static void rpc_wake_up_task_on_wq_queue_locked(struct workqueue_struct *wq, - struct rpc_wait_queue *queue, struct rpc_task *task) +static struct rpc_task * +rpc_wake_up_task_on_wq_queue_action_locked(struct workqueue_struct *wq, + struct rpc_wait_queue *queue, struct rpc_task *task, + bool (*action)(struct rpc_task *, void *), void *data) { if (RPC_IS_QUEUED(task)) { smp_rmb(); - if (task->tk_waitqueue == queue) - __rpc_do_wake_up_task_on_wq(wq, queue, task); + if (task->tk_waitqueue == queue) { + if (action == NULL || action(task, data)) { + __rpc_do_wake_up_task_on_wq(wq, queue, task); + return task; + } + } } + return NULL; +} + +static void +rpc_wake_up_task_on_wq_queue_locked(struct workqueue_struct *wq, + struct rpc_wait_queue *queue, struct rpc_task *task) +{ + rpc_wake_up_task_on_wq_queue_action_locked(wq, queue, task, NULL, NULL); } /* @@ -481,6 +495,38 @@ void rpc_wake_up_queued_task(struct rpc_wait_queue *queue, struct rpc_task *task } EXPORT_SYMBOL_GPL(rpc_wake_up_queued_task); +static bool rpc_task_action_set_status(struct rpc_task *task, void *status) +{ + task->tk_status = *(int *)status; + return true; +} + +static void +rpc_wake_up_task_queue_set_status_locked(struct rpc_wait_queue *queue, + struct rpc_task *task, int status) +{ + rpc_wake_up_task_on_wq_queue_action_locked(rpciod_workqueue, queue, + task, rpc_task_action_set_status, &status); +} + +/** + * rpc_wake_up_queued_task_set_status - wake up a task and set task->tk_status + * @queue: pointer to rpc_wait_queue + * @task: pointer to rpc_task + * @status: integer error value + * + * If @task is queued on @queue, then it is woken up, and @task->tk_status is + * set to the value of @status. + */ +void +rpc_wake_up_queued_task_set_status(struct rpc_wait_queue *queue, + struct rpc_task *task, int status) +{ + spin_lock_bh(&queue->lock); + rpc_wake_up_task_queue_set_status_locked(queue, task, status); + spin_unlock_bh(&queue->lock); +} + /* * Wake up the next task on a priority queue. */ @@ -553,12 +599,9 @@ struct rpc_task *rpc_wake_up_first_on_wq(struct workqueue_struct *wq, queue, rpc_qname(queue)); spin_lock_bh(&queue->lock); task = __rpc_find_next_queued(queue); - if (task != NULL) { - if (func(task, data)) - rpc_wake_up_task_on_wq_queue_locked(wq, queue, task); - else - task = NULL; - } + if (task != NULL) + task = rpc_wake_up_task_on_wq_queue_action_locked(wq, queue, + task, func, data); spin_unlock_bh(&queue->lock); return task; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 12/34] SUNRPC: Don't wake queued RPC calls multiple times in xprt_transmit 2018-09-04 21:05 ` [PATCH v2 11/34] SUNRPC: Add a helper to wake up a sleeping rpc_task and set its status Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 13/34] SUNRPC: Rename xprt->recv_lock to xprt->queue_lock Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Rather than waking up the entire queue of RPC messages a second time, just wake up the task that was put to sleep. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprt.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index bf8fc1a5dbd1..0441e6c8153d 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1075,13 +1075,10 @@ void xprt_transmit(struct rpc_task *task) spin_lock(&xprt->recv_lock); if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { rpc_sleep_on(&xprt->pending, task, xprt_timer); - /* - * Send an extra queue wakeup call if the - * connection was dropped in case the call to - * rpc_sleep_on() raced. - */ + /* Wake up immediately if the connection was dropped */ if (!xprt_connected(xprt)) - xprt_wake_pending_tasks(xprt, -ENOTCONN); + rpc_wake_up_queued_task_set_status(&xprt->pending, + task, -ENOTCONN); } spin_unlock(&xprt->recv_lock); } -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 13/34] SUNRPC: Rename xprt->recv_lock to xprt->queue_lock 2018-09-04 21:05 ` [PATCH v2 12/34] SUNRPC: Don't wake queued RPC calls multiple times in xprt_transmit Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 14/34] SUNRPC: Refactor xprt_transmit() to remove the reply queue code Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs We will use the same lock to protect both the transmit and receive queues. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 2 +- net/sunrpc/svcsock.c | 6 ++--- net/sunrpc/xprt.c | 24 ++++++++--------- net/sunrpc/xprtrdma/rpc_rdma.c | 10 ++++---- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 4 +-- net/sunrpc/xprtsock.c | 30 +++++++++++----------- 6 files changed, 38 insertions(+), 38 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index bd743c51a865..c25d0a5fda69 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -235,7 +235,7 @@ struct rpc_xprt { */ spinlock_t transport_lock; /* lock transport info */ spinlock_t reserve_lock; /* lock slot table */ - spinlock_t recv_lock; /* lock receive list */ + spinlock_t queue_lock; /* send/receive queue lock */ u32 xid; /* Next XID value to use */ struct rpc_task * snd_task; /* Task blocked in send */ struct svc_xprt *bc_xprt; /* NFSv4.1 backchannel */ diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 5445145e639c..db8bb6b3a2b0 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1004,7 +1004,7 @@ static int receive_cb_reply(struct svc_sock *svsk, struct svc_rqst *rqstp) if (!bc_xprt) return -EAGAIN; - spin_lock(&bc_xprt->recv_lock); + spin_lock(&bc_xprt->queue_lock); req = xprt_lookup_rqst(bc_xprt, xid); if (!req) goto unlock_notfound; @@ -1022,7 +1022,7 @@ static int receive_cb_reply(struct svc_sock *svsk, struct svc_rqst *rqstp) memcpy(dst->iov_base, src->iov_base, src->iov_len); xprt_complete_rqst(req->rq_task, rqstp->rq_arg.len); rqstp->rq_arg.len = 0; - spin_unlock(&bc_xprt->recv_lock); + spin_unlock(&bc_xprt->queue_lock); return 0; unlock_notfound: printk(KERN_NOTICE @@ -1031,7 +1031,7 @@ static int receive_cb_reply(struct svc_sock *svsk, struct svc_rqst *rqstp) __func__, ntohl(calldir), bc_xprt, ntohl(xid)); unlock_eagain: - spin_unlock(&bc_xprt->recv_lock); + spin_unlock(&bc_xprt->queue_lock); return -EAGAIN; } diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 0441e6c8153d..eda305de9f77 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -826,7 +826,7 @@ static void xprt_connect_status(struct rpc_task *task) * @xprt: transport on which the original request was transmitted * @xid: RPC XID of incoming reply * - * Caller holds xprt->recv_lock. + * Caller holds xprt->queue_lock. */ struct rpc_rqst *xprt_lookup_rqst(struct rpc_xprt *xprt, __be32 xid) { @@ -888,7 +888,7 @@ static void xprt_wait_on_pinned_rqst(struct rpc_rqst *req) * xprt_update_rtt - Update RPC RTT statistics * @task: RPC request that recently completed * - * Caller holds xprt->recv_lock. + * Caller holds xprt->queue_lock. */ void xprt_update_rtt(struct rpc_task *task) { @@ -910,7 +910,7 @@ EXPORT_SYMBOL_GPL(xprt_update_rtt); * @task: RPC request that recently completed * @copied: actual number of bytes received from the transport * - * Caller holds xprt->recv_lock. + * Caller holds xprt->queue_lock. */ void xprt_complete_rqst(struct rpc_task *task, int copied) { @@ -1030,10 +1030,10 @@ void xprt_transmit(struct rpc_task *task) memcpy(&req->rq_private_buf, &req->rq_rcv_buf, sizeof(req->rq_private_buf)); /* Add request to the receive list */ - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); list_add_tail(&req->rq_list, &xprt->recv); set_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); xprt_reset_majortimeo(req); /* Turn off autodisconnect */ del_singleshot_timer_sync(&xprt->timer); @@ -1072,7 +1072,7 @@ void xprt_transmit(struct rpc_task *task) * The spinlock ensures atomicity between the test of * req->rq_reply_bytes_recvd, and the call to rpc_sleep_on(). */ - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { rpc_sleep_on(&xprt->pending, task, xprt_timer); /* Wake up immediately if the connection was dropped */ @@ -1080,7 +1080,7 @@ void xprt_transmit(struct rpc_task *task) rpc_wake_up_queued_task_set_status(&xprt->pending, task, -ENOTCONN); } - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); } } @@ -1375,16 +1375,16 @@ void xprt_release(struct rpc_task *task) task->tk_ops->rpc_count_stats(task, task->tk_calldata); else if (task->tk_client) rpc_count_iostats(task, task->tk_client->cl_metrics); - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); if (!list_empty(&req->rq_list)) { list_del_init(&req->rq_list); if (atomic_read(&req->rq_pin)) { - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); xprt_wait_on_pinned_rqst(req); - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); } } - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); spin_lock_bh(&xprt->transport_lock); xprt->ops->release_xprt(xprt, task); if (xprt->ops->release_request) @@ -1414,7 +1414,7 @@ static void xprt_init(struct rpc_xprt *xprt, struct net *net) spin_lock_init(&xprt->transport_lock); spin_lock_init(&xprt->reserve_lock); - spin_lock_init(&xprt->recv_lock); + spin_lock_init(&xprt->queue_lock); INIT_LIST_HEAD(&xprt->free); INIT_LIST_HEAD(&xprt->recv); diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index c8ae983c6cc0..0020dc401215 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -1238,7 +1238,7 @@ void rpcrdma_complete_rqst(struct rpcrdma_rep *rep) goto out_badheader; out: - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); cwnd = xprt->cwnd; xprt->cwnd = r_xprt->rx_buf.rb_credits << RPC_CWNDSHIFT; if (xprt->cwnd > cwnd) @@ -1246,7 +1246,7 @@ void rpcrdma_complete_rqst(struct rpcrdma_rep *rep) xprt_complete_rqst(rqst->rq_task, status); xprt_unpin_rqst(rqst); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); return; /* If the incoming reply terminated a pending RPC, the next @@ -1345,7 +1345,7 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep) /* Match incoming rpcrdma_rep to an rpcrdma_req to * get context for handling any incoming chunks. */ - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); rqst = xprt_lookup_rqst(xprt, rep->rr_xid); if (!rqst) goto out_norqst; @@ -1357,7 +1357,7 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep) credits = buf->rb_max_requests; buf->rb_credits = credits; - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); req = rpcr_to_rdmar(rqst); req->rl_reply = rep; @@ -1378,7 +1378,7 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep) * is corrupt. */ out_norqst: - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); trace_xprtrdma_reply_rqst(rep); goto repost; diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index a68180090554..09b12b7568fe 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -56,7 +56,7 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, if (src->iov_len < 24) goto out_shortreply; - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); req = xprt_lookup_rqst(xprt, xid); if (!req) goto out_notfound; @@ -86,7 +86,7 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, rcvbuf->len = 0; out_unlock: - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); out: return ret; diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 3fbccebd0b10..8d6404259ff9 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -966,12 +966,12 @@ static void xs_local_data_read_skb(struct rpc_xprt *xprt, return; /* Look up and lock the request corresponding to the given XID */ - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); rovr = xprt_lookup_rqst(xprt, *xp); if (!rovr) goto out_unlock; xprt_pin_rqst(rovr); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); task = rovr->rq_task; copied = rovr->rq_private_buf.buflen; @@ -980,16 +980,16 @@ static void xs_local_data_read_skb(struct rpc_xprt *xprt, if (xs_local_copy_to_xdr(&rovr->rq_private_buf, skb)) { dprintk("RPC: sk_buff copy failed\n"); - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); goto out_unpin; } - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); xprt_complete_rqst(task, copied); out_unpin: xprt_unpin_rqst(rovr); out_unlock: - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); } static void xs_local_data_receive(struct sock_xprt *transport) @@ -1058,13 +1058,13 @@ static void xs_udp_data_read_skb(struct rpc_xprt *xprt, return; /* Look up and lock the request corresponding to the given XID */ - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); rovr = xprt_lookup_rqst(xprt, *xp); if (!rovr) goto out_unlock; xprt_pin_rqst(rovr); xprt_update_rtt(rovr->rq_task); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); task = rovr->rq_task; if ((copied = rovr->rq_private_buf.buflen) > repsize) @@ -1072,7 +1072,7 @@ static void xs_udp_data_read_skb(struct rpc_xprt *xprt, /* Suck it into the iovec, verify checksum if not done by hw. */ if (csum_partial_copy_to_xdr(&rovr->rq_private_buf, skb)) { - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); __UDPX_INC_STATS(sk, UDP_MIB_INERRORS); goto out_unpin; } @@ -1081,13 +1081,13 @@ static void xs_udp_data_read_skb(struct rpc_xprt *xprt, spin_lock_bh(&xprt->transport_lock); xprt_adjust_cwnd(xprt, task, copied); spin_unlock_bh(&xprt->transport_lock); - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); xprt_complete_rqst(task, copied); __UDPX_INC_STATS(sk, UDP_MIB_INDATAGRAMS); out_unpin: xprt_unpin_rqst(rovr); out_unlock: - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); } static void xs_udp_data_receive(struct sock_xprt *transport) @@ -1356,24 +1356,24 @@ static inline int xs_tcp_read_reply(struct rpc_xprt *xprt, dprintk("RPC: read reply XID %08x\n", ntohl(transport->recv.xid)); /* Find and lock the request corresponding to this xid */ - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); req = xprt_lookup_rqst(xprt, transport->recv.xid); if (!req) { dprintk("RPC: XID %08x request not found!\n", ntohl(transport->recv.xid)); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); return -1; } xprt_pin_rqst(req); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); xs_tcp_read_common(xprt, desc, req); - spin_lock(&xprt->recv_lock); + spin_lock(&xprt->queue_lock); if (!(transport->recv.flags & TCP_RCV_COPY_DATA)) xprt_complete_rqst(req->rq_task, transport->recv.copied); xprt_unpin_rqst(req); - spin_unlock(&xprt->recv_lock); + spin_unlock(&xprt->queue_lock); return 0; } -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 14/34] SUNRPC: Refactor xprt_transmit() to remove the reply queue code 2018-09-04 21:05 ` [PATCH v2 13/34] SUNRPC: Rename xprt->recv_lock to xprt->queue_lock Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 15/34] SUNRPC: Refactor xprt_transmit() to remove wait for reply code Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Separate out the action of adding a request to the reply queue so that the backchannel code can simply skip calling it altogether. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/clnt.c | 5 ++ net/sunrpc/xprt.c | 100 ++++++++++++++++++++++-------------- 3 files changed, 68 insertions(+), 38 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index c25d0a5fda69..0250294c904a 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -334,6 +334,7 @@ void xprt_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *req); void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task); bool xprt_prepare_transmit(struct rpc_task *task); +void xprt_request_enqueue_receive(struct rpc_task *task); void xprt_transmit(struct rpc_task *task); void xprt_end_transmit(struct rpc_task *task); int xprt_adjust_timeout(struct rpc_rqst *req); diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index a858366cd15d..414966273a3f 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1962,6 +1962,11 @@ call_transmit(struct rpc_task *task) return; } } + + /* Add task to reply queue before transmission to avoid races */ + if (rpc_reply_expected(task)) + xprt_request_enqueue_receive(task); + if (!xprt_prepare_transmit(task)) return; task->tk_action = call_transmit_status; diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index eda305de9f77..6a5f2dbdc00f 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -884,6 +884,57 @@ static void xprt_wait_on_pinned_rqst(struct rpc_rqst *req) wait_var_event(&req->rq_pin, !xprt_is_pinned_rqst(req)); } +static bool +xprt_request_data_received(struct rpc_task *task) +{ + return !test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate) && + READ_ONCE(task->tk_rqstp->rq_reply_bytes_recvd) != 0; +} + +/** + * xprt_request_enqueue_receive - Add an request to the receive queue + * @task: RPC task + * + */ +void +xprt_request_enqueue_receive(struct rpc_task *task) +{ + struct rpc_rqst *req = task->tk_rqstp; + struct rpc_xprt *xprt = req->rq_xprt; + + spin_lock(&xprt->queue_lock); + if (xprt_request_data_received(task) || !list_empty(&req->rq_list)) { + spin_unlock(&xprt->queue_lock); + return; + } + + /* Update the softirq receive buffer */ + memcpy(&req->rq_private_buf, &req->rq_rcv_buf, + sizeof(req->rq_private_buf)); + + /* Add request to the receive list */ + list_add_tail(&req->rq_list, &xprt->recv); + set_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); + spin_unlock(&xprt->queue_lock); + + xprt_reset_majortimeo(req); + /* Turn off autodisconnect */ + del_singleshot_timer_sync(&xprt->timer); +} + +/** + * xprt_request_dequeue_receive_locked - Remove a request from the receive queue + * @task: RPC task + * + * Caller must hold xprt->queue_lock. + */ +static void +xprt_request_dequeue_receive_locked(struct rpc_task *task) +{ + clear_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); + list_del_init(&task->tk_rqstp->rq_list); +} + /** * xprt_update_rtt - Update RPC RTT statistics * @task: RPC request that recently completed @@ -923,24 +974,16 @@ void xprt_complete_rqst(struct rpc_task *task, int copied) xprt->stat.recvs++; - list_del_init(&req->rq_list); req->rq_private_buf.len = copied; /* Ensure all writes are done before we update */ /* req->rq_reply_bytes_recvd */ smp_wmb(); req->rq_reply_bytes_recvd = copied; - clear_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); + xprt_request_dequeue_receive_locked(task); rpc_wake_up_queued_task(&xprt->pending, task); } EXPORT_SYMBOL_GPL(xprt_complete_rqst); -static bool -xprt_request_data_received(struct rpc_task *task) -{ - return !test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate) && - task->tk_rqstp->rq_reply_bytes_recvd != 0; -} - static void xprt_timer(struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; @@ -1014,32 +1057,15 @@ void xprt_transmit(struct rpc_task *task) dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, req->rq_slen); - if (!req->rq_reply_bytes_recvd) { - + if (!req->rq_bytes_sent) { + if (xprt_request_data_received(task)) + return; /* Verify that our message lies in the RPCSEC_GSS window */ - if (!req->rq_bytes_sent && rpcauth_xmit_need_reencode(task)) { + if (rpcauth_xmit_need_reencode(task)) { task->tk_status = -EBADMSG; return; } - - if (list_empty(&req->rq_list) && rpc_reply_expected(task)) { - /* - * Add to the list only if we're expecting a reply - */ - /* Update the softirq receive buffer */ - memcpy(&req->rq_private_buf, &req->rq_rcv_buf, - sizeof(req->rq_private_buf)); - /* Add request to the receive list */ - spin_lock(&xprt->queue_lock); - list_add_tail(&req->rq_list, &xprt->recv); - set_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); - spin_unlock(&xprt->queue_lock); - xprt_reset_majortimeo(req); - /* Turn off autodisconnect */ - del_singleshot_timer_sync(&xprt->timer); - } - } else if (xprt_request_data_received(task) && !req->rq_bytes_sent) - return; + } connect_cookie = xprt->connect_cookie; status = xprt->ops->send_request(task); @@ -1376,13 +1402,11 @@ void xprt_release(struct rpc_task *task) else if (task->tk_client) rpc_count_iostats(task, task->tk_client->cl_metrics); spin_lock(&xprt->queue_lock); - if (!list_empty(&req->rq_list)) { - list_del_init(&req->rq_list); - if (atomic_read(&req->rq_pin)) { - spin_unlock(&xprt->queue_lock); - xprt_wait_on_pinned_rqst(req); - spin_lock(&xprt->queue_lock); - } + xprt_request_dequeue_receive_locked(task); + while (xprt_is_pinned_rqst(req)) { + spin_unlock(&xprt->queue_lock); + xprt_wait_on_pinned_rqst(req); + spin_lock(&xprt->queue_lock); } spin_unlock(&xprt->queue_lock); spin_lock_bh(&xprt->transport_lock); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 15/34] SUNRPC: Refactor xprt_transmit() to remove wait for reply code 2018-09-04 21:05 ` [PATCH v2 14/34] SUNRPC: Refactor xprt_transmit() to remove the reply queue code Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 16/34] SUNRPC: Minor cleanup for call_transmit() Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Allow the caller in clnt.c to call into the code to wait for a reply after calling xprt_transmit(). Again, the reason is that the backchannel code does not need this functionality. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/clnt.c | 10 +----- net/sunrpc/xprt.c | 72 ++++++++++++++++++++++++++----------- 3 files changed, 53 insertions(+), 30 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 0250294c904a..4fa2af087cff 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -335,6 +335,7 @@ void xprt_free_slot(struct rpc_xprt *xprt, void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task); bool xprt_prepare_transmit(struct rpc_task *task); void xprt_request_enqueue_receive(struct rpc_task *task); +void xprt_request_wait_receive(struct rpc_task *task); void xprt_transmit(struct rpc_task *task); void xprt_end_transmit(struct rpc_task *task); int xprt_adjust_timeout(struct rpc_rqst *req); diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 414966273a3f..775d6e80b6e8 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1975,15 +1975,6 @@ call_transmit(struct rpc_task *task) return; if (is_retrans) task->tk_client->cl_stats->rpcretrans++; - /* - * On success, ensure that we call xprt_end_transmit() before sleeping - * in order to allow access to the socket to other RPC requests. - */ - call_transmit_status(task); - if (rpc_reply_expected(task)) - return; - task->tk_action = rpc_exit_task; - rpc_wake_up_queued_task(&task->tk_rqstp->rq_xprt->pending, task); } /* @@ -2000,6 +1991,7 @@ call_transmit_status(struct rpc_task *task) */ if (task->tk_status == 0) { xprt_end_transmit(task); + xprt_request_wait_receive(task); return; } diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 6a5f2dbdc00f..1fba837e5390 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -654,6 +654,22 @@ void xprt_force_disconnect(struct rpc_xprt *xprt) } EXPORT_SYMBOL_GPL(xprt_force_disconnect); +static unsigned int +xprt_connect_cookie(struct rpc_xprt *xprt) +{ + return READ_ONCE(xprt->connect_cookie); +} + +static bool +xprt_request_retransmit_after_disconnect(struct rpc_task *task) +{ + struct rpc_rqst *req = task->tk_rqstp; + struct rpc_xprt *xprt = req->rq_xprt; + + return req->rq_connect_cookie != xprt_connect_cookie(xprt) || + !xprt_connected(xprt); +} + /** * xprt_conditional_disconnect - force a transport to disconnect * @xprt: transport to disconnect @@ -1000,6 +1016,39 @@ static void xprt_timer(struct rpc_task *task) task->tk_status = 0; } +/** + * xprt_request_wait_receive - wait for the reply to an RPC request + * @task: RPC task about to send a request + * + */ +void xprt_request_wait_receive(struct rpc_task *task) +{ + struct rpc_rqst *req = task->tk_rqstp; + struct rpc_xprt *xprt = req->rq_xprt; + + if (!test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) + return; + /* + * Sleep on the pending queue if we're expecting a reply. + * The spinlock ensures atomicity between the test of + * req->rq_reply_bytes_recvd, and the call to rpc_sleep_on(). + */ + spin_lock(&xprt->queue_lock); + if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { + xprt->ops->set_retrans_timeout(task); + rpc_sleep_on(&xprt->pending, task, xprt_timer); + /* + * Send an extra queue wakeup call if the + * connection was dropped in case the call to + * rpc_sleep_on() raced. + */ + if (xprt_request_retransmit_after_disconnect(task)) + rpc_wake_up_queued_task_set_status(&xprt->pending, + task, -ENOTCONN); + } + spin_unlock(&xprt->queue_lock); +} + /** * xprt_prepare_transmit - reserve the transport before sending a request * @task: RPC task about to send a request @@ -1019,9 +1068,8 @@ bool xprt_prepare_transmit(struct rpc_task *task) task->tk_status = req->rq_reply_bytes_recvd; goto out_unlock; } - if ((task->tk_flags & RPC_TASK_NO_RETRANS_TIMEOUT) - && xprt_connected(xprt) - && req->rq_connect_cookie == xprt->connect_cookie) { + if ((task->tk_flags & RPC_TASK_NO_RETRANS_TIMEOUT) && + !xprt_request_retransmit_after_disconnect(task)) { xprt->ops->set_retrans_timeout(task); rpc_sleep_on(&xprt->pending, task, xprt_timer); goto out_unlock; @@ -1082,8 +1130,6 @@ void xprt_transmit(struct rpc_task *task) task->tk_flags |= RPC_TASK_SENT; spin_lock_bh(&xprt->transport_lock); - xprt->ops->set_retrans_timeout(task); - xprt->stat.sends++; xprt->stat.req_u += xprt->stat.sends - xprt->stat.recvs; xprt->stat.bklog_u += xprt->backlog.qlen; @@ -1092,22 +1138,6 @@ void xprt_transmit(struct rpc_task *task) spin_unlock_bh(&xprt->transport_lock); req->rq_connect_cookie = connect_cookie; - if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { - /* - * Sleep on the pending queue if we're expecting a reply. - * The spinlock ensures atomicity between the test of - * req->rq_reply_bytes_recvd, and the call to rpc_sleep_on(). - */ - spin_lock(&xprt->queue_lock); - if (test_bit(RPC_TASK_NEED_RECV, &task->tk_runstate)) { - rpc_sleep_on(&xprt->pending, task, xprt_timer); - /* Wake up immediately if the connection was dropped */ - if (!xprt_connected(xprt)) - rpc_wake_up_queued_task_set_status(&xprt->pending, - task, -ENOTCONN); - } - spin_unlock(&xprt->queue_lock); - } } static void xprt_add_backlog(struct rpc_xprt *xprt, struct rpc_task *task) -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 16/34] SUNRPC: Minor cleanup for call_transmit() 2018-09-04 21:05 ` [PATCH v2 15/34] SUNRPC: Refactor xprt_transmit() to remove wait for reply code Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 17/34] SUNRPC: Distinguish between the slot allocation list and receive queue Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 775d6e80b6e8..5fbd9875544e 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1946,9 +1946,7 @@ call_transmit(struct rpc_task *task) dprint_status(task); - task->tk_action = call_status; - if (task->tk_status < 0) - return; + task->tk_action = call_transmit_status; /* Encode here so that rpcsec_gss can use correct sequence number. */ if (rpc_task_need_encode(task)) { rpc_xdr_encode(task); @@ -1969,7 +1967,6 @@ call_transmit(struct rpc_task *task) if (!xprt_prepare_transmit(task)) return; - task->tk_action = call_transmit_status; xprt_transmit(task); if (task->tk_status < 0) return; @@ -1996,19 +1993,28 @@ call_transmit_status(struct rpc_task *task) } switch (task->tk_status) { - case -EAGAIN: - case -ENOBUFS: - break; default: dprint_status(task); xprt_end_transmit(task); break; + case -EBADMSG: + clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); + task->tk_action = call_transmit; + xprt_end_transmit(task); + break; /* * Special cases: if we've been waiting on the * socket's write_space() callback, or if the * socket just returned a connection error, * then hold onto the transport lock. */ + case -ENOBUFS: + rpc_delay(task, HZ>>2); + /* fall through */ + case -EAGAIN: + task->tk_action = call_transmit; + task->tk_status = 0; + break; case -ECONNREFUSED: case -EHOSTDOWN: case -ENETDOWN: @@ -2163,22 +2169,13 @@ call_status(struct rpc_task *task) /* fall through */ case -EPIPE: case -ENOTCONN: - task->tk_action = call_bind; - break; - case -ENOBUFS: - rpc_delay(task, HZ>>2); - /* fall through */ case -EAGAIN: - task->tk_action = call_transmit; + task->tk_action = call_bind; break; case -EIO: /* shutdown or soft timeout */ rpc_exit(task, status); break; - case -EBADMSG: - clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); - task->tk_action = call_transmit; - break; default: if (clnt->cl_chatty) printk("%s: RPC call returned error %d\n", -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 17/34] SUNRPC: Distinguish between the slot allocation list and receive queue 2018-09-04 21:05 ` [PATCH v2 16/34] SUNRPC: Minor cleanup for call_transmit() Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 18/34] NFS: Add a transmission queue for RPC requests Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs When storing a struct rpc_rqst on the slot allocation list, we currently use the same field 'rq_list' as we use to store the request on the receive queue. Since the structure is never on both lists at the same time, this is OK. However, for clarity, let's make that a union with different names for the different lists so that we can more easily distinguish between the two states. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 9 +++++++-- net/sunrpc/backchannel_rqst.c | 2 +- net/sunrpc/xprt.c | 16 ++++++++-------- net/sunrpc/xprtrdma/backchannel.c | 2 +- 4 files changed, 17 insertions(+), 12 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 4fa2af087cff..9cec2d0811f2 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -82,7 +82,11 @@ struct rpc_rqst { struct page **rq_enc_pages; /* scratch pages for use by gss privacy code */ void (*rq_release_snd_buf)(struct rpc_rqst *); /* release rq_enc_pages */ - struct list_head rq_list; + + union { + struct list_head rq_list; /* Slot allocation list */ + struct list_head rq_recv; /* Receive queue */ + }; void *rq_buffer; /* Call XDR encode buffer */ size_t rq_callsize; @@ -249,7 +253,8 @@ struct rpc_xprt { struct list_head bc_pa_list; /* List of preallocated * backchannel rpc_rqst's */ #endif /* CONFIG_SUNRPC_BACKCHANNEL */ - struct list_head recv; + + struct list_head recv_queue; /* Receive queue */ struct { unsigned long bind_count, /* total number of binds */ diff --git a/net/sunrpc/backchannel_rqst.c b/net/sunrpc/backchannel_rqst.c index 3c15a99b9700..92e9ad30ec2f 100644 --- a/net/sunrpc/backchannel_rqst.c +++ b/net/sunrpc/backchannel_rqst.c @@ -91,7 +91,7 @@ struct rpc_rqst *xprt_alloc_bc_req(struct rpc_xprt *xprt, gfp_t gfp_flags) return NULL; req->rq_xprt = xprt; - INIT_LIST_HEAD(&req->rq_list); + INIT_LIST_HEAD(&req->rq_recv); INIT_LIST_HEAD(&req->rq_bc_list); /* Preallocate one XDR receive buffer */ diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 1fba837e5390..7f53e97a624f 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -708,7 +708,7 @@ static void xprt_schedule_autodisconnect(struct rpc_xprt *xprt) __must_hold(&xprt->transport_lock) { - if (list_empty(&xprt->recv) && xprt_has_timer(xprt)) + if (list_empty(&xprt->recv_queue) && xprt_has_timer(xprt)) mod_timer(&xprt->timer, xprt->last_used + xprt->idle_timeout); } @@ -718,7 +718,7 @@ xprt_init_autodisconnect(struct timer_list *t) struct rpc_xprt *xprt = from_timer(xprt, t, timer); spin_lock(&xprt->transport_lock); - if (!list_empty(&xprt->recv)) + if (!list_empty(&xprt->recv_queue)) goto out_abort; /* Reset xprt->last_used to avoid connect/autodisconnect cycling */ xprt->last_used = jiffies; @@ -848,7 +848,7 @@ struct rpc_rqst *xprt_lookup_rqst(struct rpc_xprt *xprt, __be32 xid) { struct rpc_rqst *entry; - list_for_each_entry(entry, &xprt->recv, rq_list) + list_for_each_entry(entry, &xprt->recv_queue, rq_recv) if (entry->rq_xid == xid) { trace_xprt_lookup_rqst(xprt, xid, 0); entry->rq_rtt = ktime_sub(ktime_get(), entry->rq_xtime); @@ -919,7 +919,7 @@ xprt_request_enqueue_receive(struct rpc_task *task) struct rpc_xprt *xprt = req->rq_xprt; spin_lock(&xprt->queue_lock); - if (xprt_request_data_received(task) || !list_empty(&req->rq_list)) { + if (xprt_request_data_received(task) || !list_empty(&req->rq_recv)) { spin_unlock(&xprt->queue_lock); return; } @@ -929,7 +929,7 @@ xprt_request_enqueue_receive(struct rpc_task *task) sizeof(req->rq_private_buf)); /* Add request to the receive list */ - list_add_tail(&req->rq_list, &xprt->recv); + list_add_tail(&req->rq_recv, &xprt->recv_queue); set_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); spin_unlock(&xprt->queue_lock); @@ -948,7 +948,7 @@ static void xprt_request_dequeue_receive_locked(struct rpc_task *task) { clear_bit(RPC_TASK_NEED_RECV, &task->tk_runstate); - list_del_init(&task->tk_rqstp->rq_list); + list_del_init(&task->tk_rqstp->rq_recv); } /** @@ -1337,7 +1337,7 @@ xprt_request_init(struct rpc_task *task) struct rpc_xprt *xprt = task->tk_xprt; struct rpc_rqst *req = task->tk_rqstp; - INIT_LIST_HEAD(&req->rq_list); + INIT_LIST_HEAD(&req->rq_recv); req->rq_timeout = task->tk_client->cl_timeout->to_initval; req->rq_task = task; req->rq_xprt = xprt; @@ -1471,7 +1471,7 @@ static void xprt_init(struct rpc_xprt *xprt, struct net *net) spin_lock_init(&xprt->queue_lock); INIT_LIST_HEAD(&xprt->free); - INIT_LIST_HEAD(&xprt->recv); + INIT_LIST_HEAD(&xprt->recv_queue); #if defined(CONFIG_SUNRPC_BACKCHANNEL) spin_lock_init(&xprt->bc_pa_lock); INIT_LIST_HEAD(&xprt->bc_pa_list); diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c index 90adeff4c06b..40c7c7306a99 100644 --- a/net/sunrpc/xprtrdma/backchannel.c +++ b/net/sunrpc/xprtrdma/backchannel.c @@ -51,7 +51,7 @@ static int rpcrdma_bc_setup_reqs(struct rpcrdma_xprt *r_xprt, rqst = &req->rl_slot; rqst->rq_xprt = xprt; - INIT_LIST_HEAD(&rqst->rq_list); + INIT_LIST_HEAD(&rqst->rq_recv); INIT_LIST_HEAD(&rqst->rq_bc_list); __set_bit(RPC_BC_PA_IN_USE, &rqst->rq_bc_pa_state); spin_lock_bh(&xprt->bc_pa_lock); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 18/34] NFS: Add a transmission queue for RPC requests 2018-09-04 21:05 ` [PATCH v2 17/34] SUNRPC: Distinguish between the slot allocation list and receive queue Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 19/34] SUNRPC: Refactor RPC call encoding Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 6 +++ net/sunrpc/backchannel_rqst.c | 1 + net/sunrpc/clnt.c | 6 +-- net/sunrpc/xprt.c | 74 +++++++++++++++++++++++++++---- net/sunrpc/xprtrdma/backchannel.c | 1 + 5 files changed, 77 insertions(+), 11 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 9cec2d0811f2..81a6c2c8dfc7 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -88,6 +88,8 @@ struct rpc_rqst { struct list_head rq_recv; /* Receive queue */ }; + struct list_head rq_xmit; /* Send queue */ + void *rq_buffer; /* Call XDR encode buffer */ size_t rq_callsize; void *rq_rbuffer; /* Reply XDR decode buffer */ @@ -242,6 +244,9 @@ struct rpc_xprt { spinlock_t queue_lock; /* send/receive queue lock */ u32 xid; /* Next XID value to use */ struct rpc_task * snd_task; /* Task blocked in send */ + + struct list_head xmit_queue; /* Send queue */ + struct svc_xprt *bc_xprt; /* NFSv4.1 backchannel */ #if defined(CONFIG_SUNRPC_BACKCHANNEL) struct svc_serv *bc_serv; /* The RPC service which will */ @@ -339,6 +344,7 @@ void xprt_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *req); void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task); bool xprt_prepare_transmit(struct rpc_task *task); +void xprt_request_enqueue_transmit(struct rpc_task *task); void xprt_request_enqueue_receive(struct rpc_task *task); void xprt_request_wait_receive(struct rpc_task *task); void xprt_transmit(struct rpc_task *task); diff --git a/net/sunrpc/backchannel_rqst.c b/net/sunrpc/backchannel_rqst.c index 92e9ad30ec2f..39b394b7dae3 100644 --- a/net/sunrpc/backchannel_rqst.c +++ b/net/sunrpc/backchannel_rqst.c @@ -92,6 +92,7 @@ struct rpc_rqst *xprt_alloc_bc_req(struct rpc_xprt *xprt, gfp_t gfp_flags) req->rq_xprt = xprt; INIT_LIST_HEAD(&req->rq_recv); + INIT_LIST_HEAD(&req->rq_xmit); INIT_LIST_HEAD(&req->rq_bc_list); /* Preallocate one XDR receive buffer */ diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 5fbd9875544e..a817f70d6192 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1759,8 +1759,6 @@ rpc_xdr_encode(struct rpc_task *task) task->tk_status = rpcauth_wrap_req(task, encode, req, p, task->tk_msg.rpc_argp); - if (task->tk_status == 0) - set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); } /* @@ -1959,11 +1957,13 @@ call_transmit(struct rpc_task *task) rpc_exit(task, task->tk_status); return; } + set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); } /* Add task to reply queue before transmission to avoid races */ if (rpc_reply_expected(task)) xprt_request_enqueue_receive(task); + xprt_request_enqueue_transmit(task); if (!xprt_prepare_transmit(task)) return; @@ -1998,7 +1998,6 @@ call_transmit_status(struct rpc_task *task) xprt_end_transmit(task); break; case -EBADMSG: - clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); task->tk_action = call_transmit; xprt_end_transmit(task); break; @@ -2049,6 +2048,7 @@ call_bc_transmit(struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; + xprt_request_enqueue_transmit(task); if (!xprt_prepare_transmit(task)) goto out_retry; diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 7f53e97a624f..8e8c345eedf7 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1049,6 +1049,64 @@ void xprt_request_wait_receive(struct rpc_task *task) spin_unlock(&xprt->queue_lock); } +static bool +xprt_request_need_transmit(struct rpc_task *task) +{ + return !(task->tk_flags & RPC_TASK_NO_RETRANS_TIMEOUT) || + xprt_request_retransmit_after_disconnect(task); +} + +/** + * xprt_request_enqueue_transmit - queue a task for transmission + * @task: pointer to rpc_task + * + * Add a task to the transmission queue. + */ +void +xprt_request_enqueue_transmit(struct rpc_task *task) +{ + struct rpc_rqst *req = task->tk_rqstp; + struct rpc_xprt *xprt = req->rq_xprt; + + spin_lock(&xprt->queue_lock); + if (list_empty(&req->rq_xmit) && xprt_request_need_transmit(task) && + test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) + list_add_tail(&req->rq_xmit, &xprt->xmit_queue); + spin_unlock(&xprt->queue_lock); +} + +/** + * xprt_request_dequeue_transmit_locked - remove a task from the transmission queue + * @task: pointer to rpc_task + * + * Remove a task from the transmission queue + * Caller must hold xprt->queue_lock + */ +static void +xprt_request_dequeue_transmit_locked(struct rpc_task *task) +{ + xprt_task_clear_bytes_sent(task); + clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); + list_del_init(&task->tk_rqstp->rq_xmit); +} + +/** + * xprt_request_dequeue_transmit - remove a task from the transmission queue + * @task: pointer to rpc_task + * + * Remove a task from the transmission queue + */ +static void +xprt_request_dequeue_transmit(struct rpc_task *task) +{ + struct rpc_rqst *req = task->tk_rqstp; + struct rpc_xprt *xprt = req->rq_xprt; + + spin_lock(&xprt->queue_lock); + xprt_request_dequeue_transmit_locked(task); + spin_unlock(&xprt->queue_lock); +} + /** * xprt_prepare_transmit - reserve the transport before sending a request * @task: RPC task about to send a request @@ -1068,12 +1126,8 @@ bool xprt_prepare_transmit(struct rpc_task *task) task->tk_status = req->rq_reply_bytes_recvd; goto out_unlock; } - if ((task->tk_flags & RPC_TASK_NO_RETRANS_TIMEOUT) && - !xprt_request_retransmit_after_disconnect(task)) { - xprt->ops->set_retrans_timeout(task); - rpc_sleep_on(&xprt->pending, task, xprt_timer); + if (!test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) goto out_unlock; - } } if (!xprt->ops->reserve_xprt(xprt, task)) { task->tk_status = -EAGAIN; @@ -1107,11 +1161,11 @@ void xprt_transmit(struct rpc_task *task) if (!req->rq_bytes_sent) { if (xprt_request_data_received(task)) - return; + goto out_dequeue; /* Verify that our message lies in the RPCSEC_GSS window */ if (rpcauth_xmit_need_reencode(task)) { task->tk_status = -EBADMSG; - return; + goto out_dequeue; } } @@ -1126,7 +1180,6 @@ void xprt_transmit(struct rpc_task *task) xprt_inject_disconnect(xprt); dprintk("RPC: %5u xmit complete\n", task->tk_pid); - clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); task->tk_flags |= RPC_TASK_SENT; spin_lock_bh(&xprt->transport_lock); @@ -1138,6 +1191,8 @@ void xprt_transmit(struct rpc_task *task) spin_unlock_bh(&xprt->transport_lock); req->rq_connect_cookie = connect_cookie; +out_dequeue: + xprt_request_dequeue_transmit(task); } static void xprt_add_backlog(struct rpc_xprt *xprt, struct rpc_task *task) @@ -1338,6 +1393,7 @@ xprt_request_init(struct rpc_task *task) struct rpc_rqst *req = task->tk_rqstp; INIT_LIST_HEAD(&req->rq_recv); + INIT_LIST_HEAD(&req->rq_xmit); req->rq_timeout = task->tk_client->cl_timeout->to_initval; req->rq_task = task; req->rq_xprt = xprt; @@ -1433,6 +1489,7 @@ void xprt_release(struct rpc_task *task) rpc_count_iostats(task, task->tk_client->cl_metrics); spin_lock(&xprt->queue_lock); xprt_request_dequeue_receive_locked(task); + xprt_request_dequeue_transmit_locked(task); while (xprt_is_pinned_rqst(req)) { spin_unlock(&xprt->queue_lock); xprt_wait_on_pinned_rqst(req); @@ -1472,6 +1529,7 @@ static void xprt_init(struct rpc_xprt *xprt, struct net *net) INIT_LIST_HEAD(&xprt->free); INIT_LIST_HEAD(&xprt->recv_queue); + INIT_LIST_HEAD(&xprt->xmit_queue); #if defined(CONFIG_SUNRPC_BACKCHANNEL) spin_lock_init(&xprt->bc_pa_lock); INIT_LIST_HEAD(&xprt->bc_pa_list); diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c index 40c7c7306a99..fc01fdabbbce 100644 --- a/net/sunrpc/xprtrdma/backchannel.c +++ b/net/sunrpc/xprtrdma/backchannel.c @@ -52,6 +52,7 @@ static int rpcrdma_bc_setup_reqs(struct rpcrdma_xprt *r_xprt, rqst->rq_xprt = xprt; INIT_LIST_HEAD(&rqst->rq_recv); + INIT_LIST_HEAD(&rqst->rq_xmit); INIT_LIST_HEAD(&rqst->rq_bc_list); __set_bit(RPC_BC_PA_IN_USE, &rqst->rq_bc_pa_state); spin_lock_bh(&xprt->bc_pa_lock); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 19/34] SUNRPC: Refactor RPC call encoding 2018-09-04 21:05 ` [PATCH v2 18/34] NFS: Add a transmission queue for RPC requests Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Move the call encoding so that it occurs before the transport connection etc. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 67 +++++++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index a817f70d6192..497a30762a6d 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -61,6 +61,7 @@ static void call_start(struct rpc_task *task); static void call_reserve(struct rpc_task *task); static void call_reserveresult(struct rpc_task *task); static void call_allocate(struct rpc_task *task); +static void call_encode(struct rpc_task *task); static void call_decode(struct rpc_task *task); static void call_bind(struct rpc_task *task); static void call_bind_status(struct rpc_task *task); @@ -1680,7 +1681,7 @@ call_allocate(struct rpc_task *task) dprint_status(task); task->tk_status = 0; - task->tk_action = call_bind; + task->tk_action = call_encode; if (req->rq_buffer) return; @@ -1727,9 +1728,6 @@ rpc_task_need_encode(struct rpc_task *task) return test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate) == 0; } -/* - * 3. Encode arguments of an RPC call - */ static void rpc_xdr_encode(struct rpc_task *task) { @@ -1745,6 +1743,7 @@ rpc_xdr_encode(struct rpc_task *task) xdr_buf_init(&req->rq_rcv_buf, req->rq_rbuffer, req->rq_rcvsize); + req->rq_bytes_sent = 0; p = rpc_encode_header(task); if (p == NULL) { @@ -1761,6 +1760,34 @@ rpc_xdr_encode(struct rpc_task *task) task->tk_msg.rpc_argp); } +/* + * 3. Encode arguments of an RPC call + */ +static void +call_encode(struct rpc_task *task) +{ + if (!rpc_task_need_encode(task)) + goto out; + /* Encode here so that rpcsec_gss can use correct sequence number. */ + rpc_xdr_encode(task); + /* Did the encode result in an error condition? */ + if (task->tk_status != 0) { + /* Was the error nonfatal? */ + if (task->tk_status == -EAGAIN) + rpc_delay(task, HZ >> 4); + else + rpc_exit(task, task->tk_status); + return; + } + + /* Add task to reply queue before transmission to avoid races */ + if (rpc_reply_expected(task)) + xprt_request_enqueue_receive(task); + set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); +out: + task->tk_action = call_bind; +} + /* * 4. Get the server port number if not yet set */ @@ -1945,25 +1972,9 @@ call_transmit(struct rpc_task *task) dprint_status(task); task->tk_action = call_transmit_status; - /* Encode here so that rpcsec_gss can use correct sequence number. */ - if (rpc_task_need_encode(task)) { - rpc_xdr_encode(task); - /* Did the encode result in an error condition? */ - if (task->tk_status != 0) { - /* Was the error nonfatal? */ - if (task->tk_status == -EAGAIN) - rpc_delay(task, HZ >> 4); - else - rpc_exit(task, task->tk_status); - return; - } - set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); - } - - /* Add task to reply queue before transmission to avoid races */ - if (rpc_reply_expected(task)) - xprt_request_enqueue_receive(task); xprt_request_enqueue_transmit(task); + if (!test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) + return; if (!xprt_prepare_transmit(task)) return; @@ -1998,8 +2009,8 @@ call_transmit_status(struct rpc_task *task) xprt_end_transmit(task); break; case -EBADMSG: - task->tk_action = call_transmit; xprt_end_transmit(task); + task->tk_action = call_encode; break; /* * Special cases: if we've been waiting on the @@ -2170,7 +2181,7 @@ call_status(struct rpc_task *task) case -EPIPE: case -ENOTCONN: case -EAGAIN: - task->tk_action = call_bind; + task->tk_action = call_encode; break; case -EIO: /* shutdown or soft timeout */ @@ -2235,7 +2246,7 @@ call_timeout(struct rpc_task *task) rpcauth_invalcred(task); retry: - task->tk_action = call_bind; + task->tk_action = call_encode; task->tk_status = 0; } @@ -2279,7 +2290,7 @@ call_decode(struct rpc_task *task) if (req->rq_rcv_buf.len < 12) { if (!RPC_IS_SOFT(task)) { - task->tk_action = call_bind; + task->tk_action = call_encode; goto out_retry; } dprintk("RPC: %s: too small RPC reply size (%d bytes)\n", @@ -2410,7 +2421,7 @@ rpc_verify_header(struct rpc_task *task) task->tk_garb_retry--; dprintk("RPC: %5u %s: retry garbled creds\n", task->tk_pid, __func__); - task->tk_action = call_bind; + task->tk_action = call_encode; goto out_retry; case RPC_AUTH_TOOWEAK: printk(KERN_NOTICE "RPC: server %s requires stronger " @@ -2479,7 +2490,7 @@ rpc_verify_header(struct rpc_task *task) task->tk_garb_retry--; dprintk("RPC: %5u %s: retrying\n", task->tk_pid, __func__); - task->tk_action = call_bind; + task->tk_action = call_encode; out_retry: return ERR_PTR(-EAGAIN); } -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() 2018-09-04 21:05 ` [PATCH v2 19/34] SUNRPC: Refactor RPC call encoding Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 21/34] SUNRPC: Don't reset the request 'bytes_sent' counter when releasing XPRT_LOCK Trond Myklebust 2018-09-06 18:49 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Schumaker, Anna 0 siblings, 2 replies; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs When we shift to using the transmit queue, then the task that holds the write lock will not necessarily be the same as the one being transmitted. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 2 +- net/sunrpc/xprt.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 3 +-- net/sunrpc/xprtrdma/transport.c | 5 ++-- net/sunrpc/xprtsock.c | 27 +++++++++++----------- 5 files changed, 18 insertions(+), 21 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 81a6c2c8dfc7..6d91acfe0644 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -140,7 +140,7 @@ struct rpc_xprt_ops { void (*connect)(struct rpc_xprt *xprt, struct rpc_task *task); int (*buf_alloc)(struct rpc_task *task); void (*buf_free)(struct rpc_task *task); - int (*send_request)(struct rpc_task *task); + int (*send_request)(struct rpc_rqst *req, struct rpc_task *task); void (*set_retrans_timeout)(struct rpc_task *task); void (*timer)(struct rpc_xprt *xprt, struct rpc_task *task); void (*release_request)(struct rpc_task *task); diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 8e8c345eedf7..7c80f93562e5 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1170,7 +1170,7 @@ void xprt_transmit(struct rpc_task *task) } connect_cookie = xprt->connect_cookie; - status = xprt->ops->send_request(task); + status = xprt->ops->send_request(req, task); trace_xprt_transmit(xprt, req->rq_xid, status); if (status != 0) { task->tk_status = status; diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index 09b12b7568fe..d1618c70edb4 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -215,9 +215,8 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) * connection. */ static int -xprt_rdma_bc_send_request(struct rpc_task *task) +xprt_rdma_bc_send_request(struct rpc_rqst *rqst, struct rpc_task *task) { - struct rpc_rqst *rqst = task->tk_rqstp; struct svc_xprt *sxprt = rqst->rq_xprt->bc_xprt; struct svcxprt_rdma *rdma; int ret; diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 143ce2579ba9..fa684bf4d090 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -706,9 +706,8 @@ xprt_rdma_free(struct rpc_task *task) * sent. Do not try to send this message again. */ static int -xprt_rdma_send_request(struct rpc_task *task) +xprt_rdma_send_request(struct rpc_rqst *rqst, struct rpc_task *task) { - struct rpc_rqst *rqst = task->tk_rqstp; struct rpc_xprt *xprt = rqst->rq_xprt; struct rpcrdma_req *req = rpcr_to_rdmar(rqst); struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt); @@ -741,7 +740,7 @@ xprt_rdma_send_request(struct rpc_task *task) /* An RPC with no reply will throw off credit accounting, * so drop the connection to reset the credit grant. */ - if (!rpc_reply_expected(task)) + if (!rpc_reply_expected(rqst->rq_task)) goto drop_connection; return 0; diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 8d6404259ff9..b8143eded4af 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -449,12 +449,12 @@ static void xs_nospace_callback(struct rpc_task *task) /** * xs_nospace - place task on wait queue if transmit was incomplete + * @req: pointer to RPC request * @task: task to put to sleep * */ -static int xs_nospace(struct rpc_task *task) +static int xs_nospace(struct rpc_rqst *req, struct rpc_task *task) { - struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); struct sock *sk = transport->inet; @@ -513,6 +513,7 @@ static inline void xs_encode_stream_record_marker(struct xdr_buf *buf) /** * xs_local_send_request - write an RPC request to an AF_LOCAL socket + * @req: pointer to RPC request * @task: RPC task that manages the state of an RPC request * * Return values: @@ -522,9 +523,8 @@ static inline void xs_encode_stream_record_marker(struct xdr_buf *buf) * ENOTCONN: Caller needs to invoke connect logic then call again * other: Some other error occured, the request was not sent */ -static int xs_local_send_request(struct rpc_task *task) +static int xs_local_send_request(struct rpc_rqst *req, struct rpc_task *task) { - struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); @@ -569,7 +569,7 @@ static int xs_local_send_request(struct rpc_task *task) case -ENOBUFS: break; case -EAGAIN: - status = xs_nospace(task); + status = xs_nospace(req, task); break; default: dprintk("RPC: sendmsg returned unrecognized error %d\n", @@ -585,6 +585,7 @@ static int xs_local_send_request(struct rpc_task *task) /** * xs_udp_send_request - write an RPC request to a UDP socket + * @req: pointer to RPC request * @task: address of RPC task that manages the state of an RPC request * * Return values: @@ -594,9 +595,8 @@ static int xs_local_send_request(struct rpc_task *task) * ENOTCONN: Caller needs to invoke connect logic then call again * other: Some other error occurred, the request was not sent */ -static int xs_udp_send_request(struct rpc_task *task) +static int xs_udp_send_request(struct rpc_rqst *req, struct rpc_task *task) { - struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); struct xdr_buf *xdr = &req->rq_snd_buf; @@ -638,7 +638,7 @@ static int xs_udp_send_request(struct rpc_task *task) /* Should we call xs_close() here? */ break; case -EAGAIN: - status = xs_nospace(task); + status = xs_nospace(req, task); break; case -ENETUNREACH: case -ENOBUFS: @@ -658,6 +658,7 @@ static int xs_udp_send_request(struct rpc_task *task) /** * xs_tcp_send_request - write an RPC request to a TCP socket + * @req: pointer to RPC request * @task: address of RPC task that manages the state of an RPC request * * Return values: @@ -670,9 +671,8 @@ static int xs_udp_send_request(struct rpc_task *task) * XXX: In the case of soft timeouts, should we eventually give up * if sendmsg is not able to make progress? */ -static int xs_tcp_send_request(struct rpc_task *task) +static int xs_tcp_send_request(struct rpc_rqst *req, struct rpc_task *task) { - struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); struct xdr_buf *xdr = &req->rq_snd_buf; @@ -697,7 +697,7 @@ static int xs_tcp_send_request(struct rpc_task *task) * completes while the socket holds a reference to the pages, * then we may end up resending corrupted data. */ - if (task->tk_flags & RPC_TASK_SENT) + if (req->rq_task->tk_flags & RPC_TASK_SENT) zerocopy = false; if (test_bit(XPRT_SOCK_UPD_TIMEOUT, &transport->sock_state)) @@ -761,7 +761,7 @@ static int xs_tcp_send_request(struct rpc_task *task) /* Should we call xs_close() here? */ break; case -EAGAIN: - status = xs_nospace(task); + status = xs_nospace(req, task); break; case -ECONNRESET: case -ECONNREFUSED: @@ -2706,9 +2706,8 @@ static int bc_sendto(struct rpc_rqst *req) /* * The send routine. Borrows from svc_send */ -static int bc_send_request(struct rpc_task *task) +static int bc_send_request(struct rpc_rqst *req, struct rpc_task *task) { - struct rpc_rqst *req = task->tk_rqstp; struct svc_xprt *xprt; int len; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 21/34] SUNRPC: Don't reset the request 'bytes_sent' counter when releasing XPRT_LOCK 2018-09-04 21:05 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 22/34] SUNRPC: Simplify xprt_prepare_transmit() Trond Myklebust 2018-09-06 18:49 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Schumaker, Anna 1 sibling, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs If the request is still on the queue, this will be incorrect behaviour. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 4 ---- net/sunrpc/xprt.c | 14 -------------- 2 files changed, 18 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 497a30762a6d..032b7042adb6 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2132,15 +2132,11 @@ static void call_status(struct rpc_task *task) { struct rpc_clnt *clnt = task->tk_client; - struct rpc_rqst *req = task->tk_rqstp; int status; if (!task->tk_msg.rpc_proc->p_proc) trace_xprt_ping(task->tk_xprt, task->tk_status); - if (req->rq_reply_bytes_recvd > 0 && !req->rq_bytes_sent) - task->tk_status = req->rq_reply_bytes_recvd; - dprint_status(task); status = task->tk_status; diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 7c80f93562e5..305aa9570873 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -332,15 +332,6 @@ static void __xprt_lock_write_next_cong(struct rpc_xprt *xprt) xprt_clear_locked(xprt); } -static void xprt_task_clear_bytes_sent(struct rpc_task *task) -{ - if (task != NULL) { - struct rpc_rqst *req = task->tk_rqstp; - if (req != NULL) - req->rq_bytes_sent = 0; - } -} - /** * xprt_release_xprt - allow other requests to use a transport * @xprt: transport with other tasks potentially waiting @@ -351,7 +342,6 @@ static void xprt_task_clear_bytes_sent(struct rpc_task *task) void xprt_release_xprt(struct rpc_xprt *xprt, struct rpc_task *task) { if (xprt->snd_task == task) { - xprt_task_clear_bytes_sent(task); xprt_clear_locked(xprt); __xprt_lock_write_next(xprt); } @@ -369,7 +359,6 @@ EXPORT_SYMBOL_GPL(xprt_release_xprt); void xprt_release_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) { if (xprt->snd_task == task) { - xprt_task_clear_bytes_sent(task); xprt_clear_locked(xprt); __xprt_lock_write_next_cong(xprt); } @@ -742,7 +731,6 @@ bool xprt_lock_connect(struct rpc_xprt *xprt, goto out; if (xprt->snd_task != task) goto out; - xprt_task_clear_bytes_sent(task); xprt->snd_task = cookie; ret = true; out: @@ -788,7 +776,6 @@ void xprt_connect(struct rpc_task *task) xprt->ops->close(xprt); if (!xprt_connected(xprt)) { - task->tk_rqstp->rq_bytes_sent = 0; task->tk_timeout = task->tk_rqstp->rq_timeout; task->tk_rqstp->rq_connect_cookie = xprt->connect_cookie; rpc_sleep_on(&xprt->pending, task, xprt_connect_status); @@ -1085,7 +1072,6 @@ xprt_request_enqueue_transmit(struct rpc_task *task) static void xprt_request_dequeue_transmit_locked(struct rpc_task *task) { - xprt_task_clear_bytes_sent(task); clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); list_del_init(&task->tk_rqstp->rq_xmit); } -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 22/34] SUNRPC: Simplify xprt_prepare_transmit() 2018-09-04 21:05 ` [PATCH v2 21/34] SUNRPC: Don't reset the request 'bytes_sent' counter when releasing XPRT_LOCK Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Remove the checks for whether or not we need to transmit, and whether or not a reply has been received. Those are already handled in call_transmit() itself. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprt.c | 21 +++------------------ 1 file changed, 3 insertions(+), 18 deletions(-) diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 305aa9570873..f1301d391399 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1102,27 +1102,12 @@ bool xprt_prepare_transmit(struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; - bool ret = false; dprintk("RPC: %5u xprt_prepare_transmit\n", task->tk_pid); - spin_lock_bh(&xprt->transport_lock); - if (!req->rq_bytes_sent) { - if (req->rq_reply_bytes_recvd) { - task->tk_status = req->rq_reply_bytes_recvd; - goto out_unlock; - } - if (!test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) - goto out_unlock; - } - if (!xprt->ops->reserve_xprt(xprt, task)) { - task->tk_status = -EAGAIN; - goto out_unlock; - } - ret = true; -out_unlock: - spin_unlock_bh(&xprt->transport_lock); - return ret; + if (!xprt_lock_write(xprt, task)) + return false; + return true; } void xprt_end_transmit(struct rpc_task *task) -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() 2018-09-04 21:05 ` [PATCH v2 22/34] SUNRPC: Simplify xprt_prepare_transmit() Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 24/34] SUNRPC: Fix up the back channel transmit Trond Myklebust 2018-09-05 14:30 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Chuck Lever 0 siblings, 2 replies; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 6 ------ net/sunrpc/xprt.c | 13 ++++++------- 2 files changed, 6 insertions(+), 13 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 032b7042adb6..21713bed812a 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1967,8 +1967,6 @@ call_connect_status(struct rpc_task *task) static void call_transmit(struct rpc_task *task) { - int is_retrans = RPC_WAS_SENT(task); - dprint_status(task); task->tk_action = call_transmit_status; @@ -1979,10 +1977,6 @@ call_transmit(struct rpc_task *task) if (!xprt_prepare_transmit(task)) return; xprt_transmit(task); - if (task->tk_status < 0) - return; - if (is_retrans) - task->tk_client->cl_stats->rpcretrans++; } /* diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index f1301d391399..e2f5b4668a66 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -191,8 +191,6 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task) goto out_sleep; } xprt->snd_task = task; - if (req != NULL) - req->rq_ntrans++; return 1; @@ -247,7 +245,6 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) } if (__xprt_get_cong(xprt, task)) { xprt->snd_task = task; - req->rq_ntrans++; return 1; } xprt_clear_locked(xprt); @@ -281,12 +278,8 @@ static inline int xprt_lock_write(struct rpc_xprt *xprt, struct rpc_task *task) static bool __xprt_lock_write_func(struct rpc_task *task, void *data) { struct rpc_xprt *xprt = data; - struct rpc_rqst *req; - req = task->tk_rqstp; xprt->snd_task = task; - if (req) - req->rq_ntrans++; return true; } @@ -1126,6 +1119,7 @@ void xprt_transmit(struct rpc_task *task) struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; unsigned int connect_cookie; + int is_retrans = RPC_WAS_SENT(task); int status; dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, req->rq_slen); @@ -1140,6 +1134,8 @@ void xprt_transmit(struct rpc_task *task) } } + req->rq_ntrans++; + connect_cookie = xprt->connect_cookie; status = xprt->ops->send_request(req, task); trace_xprt_transmit(xprt, req->rq_xid, status); @@ -1148,6 +1144,9 @@ void xprt_transmit(struct rpc_task *task) return; } + if (is_retrans) + task->tk_client->cl_stats->rpcretrans++; + xprt_inject_disconnect(xprt); dprintk("RPC: %5u xmit complete\n", task->tk_pid); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 24/34] SUNRPC: Fix up the back channel transmit 2018-09-04 21:05 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 25/34] SUNRPC: Support for congestion control when queuing is enabled Trond Myklebust 2018-09-05 14:30 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Chuck Lever 1 sibling, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Fix up the back channel code to recognise that it has already been transmitted, so does not need to be called again. Also ensure that we set req->rq_task. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 21713bed812a..00384bde593e 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1150,6 +1150,7 @@ struct rpc_task *rpc_run_bc_task(struct rpc_rqst *req) */ task = rpc_new_task(&task_setup_data); task->tk_rqstp = req; + req->rq_task = task; /* * Set up the xdr_buf length. @@ -2054,6 +2055,9 @@ call_bc_transmit(struct rpc_task *task) struct rpc_rqst *req = task->tk_rqstp; xprt_request_enqueue_transmit(task); + if (!test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) + goto out_wakeup; + if (!xprt_prepare_transmit(task)) goto out_retry; @@ -2108,6 +2112,7 @@ call_bc_transmit(struct rpc_task *task) "error: %d\n", task->tk_status); break; } +out_wakeup: rpc_wake_up_queued_task(&req->rq_xprt->pending, task); out_done: task->tk_action = rpc_exit_task; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 25/34] SUNRPC: Support for congestion control when queuing is enabled 2018-09-04 21:05 ` [PATCH v2 24/34] SUNRPC: Fix up the back channel transmit Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 26/34] SUNRPC: Improve latency for interactive tasks Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Both RDMA and UDP transports require the request to get a "congestion control" credit before they can be transmitted. Right now, this is done when the request locks the socket. We'd like it to happen when a request attempts to be transmitted for the first time. In order to support retransmission of requests that already hold such credits, we also want to ensure that they get queued first, so that we don't deadlock with requests that have yet to obtain a credit. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/clnt.c | 5 +++ net/sunrpc/xprt.c | 70 ++++++++++++++++++------------- net/sunrpc/xprtrdma/backchannel.c | 3 ++ net/sunrpc/xprtrdma/transport.c | 3 ++ net/sunrpc/xprtsock.c | 4 ++ 6 files changed, 57 insertions(+), 29 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 6d91acfe0644..b23c757bebfc 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -395,6 +395,7 @@ void xprt_complete_rqst(struct rpc_task *task, int copied); void xprt_pin_rqst(struct rpc_rqst *req); void xprt_unpin_rqst(struct rpc_rqst *req); void xprt_release_rqst_cong(struct rpc_task *task); +bool xprt_request_get_cong(struct rpc_xprt *xprt, struct rpc_rqst *req); void xprt_disconnect_done(struct rpc_xprt *xprt); void xprt_force_disconnect(struct rpc_xprt *xprt); void xprt_conditional_disconnect(struct rpc_xprt *xprt, unsigned int cookie); diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 00384bde593e..52494baca7bc 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2003,6 +2003,11 @@ call_transmit_status(struct rpc_task *task) dprint_status(task); xprt_end_transmit(task); break; + case -EBADSLT: + xprt_end_transmit(task); + task->tk_action = call_transmit; + task->tk_status = 0; + break; case -EBADMSG: xprt_end_transmit(task); task->tk_action = call_encode; diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index e2f5b4668a66..a7a93d61567f 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -68,8 +68,6 @@ static void xprt_init(struct rpc_xprt *xprt, struct net *net); static __be32 xprt_alloc_xid(struct rpc_xprt *xprt); static void xprt_connect_status(struct rpc_task *task); -static int __xprt_get_cong(struct rpc_xprt *, struct rpc_task *); -static void __xprt_put_cong(struct rpc_xprt *, struct rpc_rqst *); static void xprt_destroy(struct rpc_xprt *xprt); static DEFINE_SPINLOCK(xprt_list_lock); @@ -228,6 +226,7 @@ static void xprt_clear_locked(struct rpc_xprt *xprt) * Same as xprt_reserve_xprt, but Van Jacobson congestion control is * integrated into the decision of whether a request is allowed to be * woken up and given access to the transport. + * Note that the lock is only granted if we know there are free slots. */ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) { @@ -243,14 +242,12 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) xprt->snd_task = task; return 1; } - if (__xprt_get_cong(xprt, task)) { + if (!RPCXPRT_CONGESTED(xprt)) { xprt->snd_task = task; return 1; } xprt_clear_locked(xprt); out_sleep: - if (req) - __xprt_put_cong(xprt, req); dprintk("RPC: %5u failed to lock transport %p\n", task->tk_pid, xprt); task->tk_timeout = 0; task->tk_status = -EAGAIN; @@ -294,24 +291,6 @@ static void __xprt_lock_write_next(struct rpc_xprt *xprt) xprt_clear_locked(xprt); } -static bool __xprt_lock_write_cong_func(struct rpc_task *task, void *data) -{ - struct rpc_xprt *xprt = data; - struct rpc_rqst *req; - - req = task->tk_rqstp; - if (req == NULL) { - xprt->snd_task = task; - return true; - } - if (__xprt_get_cong(xprt, task)) { - xprt->snd_task = task; - req->rq_ntrans++; - return true; - } - return false; -} - static void __xprt_lock_write_next_cong(struct rpc_xprt *xprt) { if (test_and_set_bit(XPRT_LOCKED, &xprt->state)) @@ -319,7 +298,7 @@ static void __xprt_lock_write_next_cong(struct rpc_xprt *xprt) if (RPCXPRT_CONGESTED(xprt)) goto out_unlock; if (rpc_wake_up_first_on_wq(xprtiod_workqueue, &xprt->sending, - __xprt_lock_write_cong_func, xprt)) + __xprt_lock_write_func, xprt)) return; out_unlock: xprt_clear_locked(xprt); @@ -370,14 +349,12 @@ static inline void xprt_release_write(struct rpc_xprt *xprt, struct rpc_task *ta * overflowed. Put the task to sleep if this is the case. */ static int -__xprt_get_cong(struct rpc_xprt *xprt, struct rpc_task *task) +__xprt_get_cong(struct rpc_xprt *xprt, struct rpc_rqst *req) { - struct rpc_rqst *req = task->tk_rqstp; - if (req->rq_cong) return 1; dprintk("RPC: %5u xprt_cwnd_limited cong = %lu cwnd = %lu\n", - task->tk_pid, xprt->cong, xprt->cwnd); + req->rq_task->tk_pid, xprt->cong, xprt->cwnd); if (RPCXPRT_CONGESTED(xprt)) return 0; req->rq_cong = 1; @@ -399,6 +376,25 @@ __xprt_put_cong(struct rpc_xprt *xprt, struct rpc_rqst *req) __xprt_lock_write_next_cong(xprt); } +/** + * xprt_request_get_cong - Request congestion control credits + * @xprt: pointer to transport + * @req: pointer to RPC request + * + * Useful for transports that require congestion control. + */ +bool +xprt_request_get_cong(struct rpc_xprt *xprt, struct rpc_rqst *req) +{ + bool ret = false; + + spin_lock_bh(&xprt->transport_lock); + ret = __xprt_get_cong(xprt, req) != 0; + spin_unlock_bh(&xprt->transport_lock); + return ret; +} +EXPORT_SYMBOL_GPL(xprt_request_get_cong); + /** * xprt_release_rqst_cong - housekeeping when request is complete * @task: RPC request that recently completed @@ -1050,8 +1046,24 @@ xprt_request_enqueue_transmit(struct rpc_task *task) spin_lock(&xprt->queue_lock); if (list_empty(&req->rq_xmit) && xprt_request_need_transmit(task) && - test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) + test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) { + /* + * Requests that carry congestion control credits are added + * to the head of the list to avoid starvation issues. + */ + if (req->rq_cong) { + struct rpc_rqst *pos; + list_for_each_entry(pos, &xprt->xmit_queue, rq_xmit) { + if (pos->rq_cong) + continue; + /* Note: req is added _before_ pos */ + list_add_tail(&req->rq_xmit, &pos->rq_xmit); + goto out; + } + } list_add_tail(&req->rq_xmit, &xprt->xmit_queue); + } +out: spin_unlock(&xprt->queue_lock); } diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c index fc01fdabbbce..14fc4596075e 100644 --- a/net/sunrpc/xprtrdma/backchannel.c +++ b/net/sunrpc/xprtrdma/backchannel.c @@ -202,6 +202,9 @@ int xprt_rdma_bc_send_reply(struct rpc_rqst *rqst) if (!xprt_connected(rqst->rq_xprt)) goto drop_connection; + if (!xprt_request_get_cong(rqst->rq_xprt, rqst)) + return -EBADSLT; + rc = rpcrdma_bc_marshal_reply(rqst); if (rc < 0) goto failed_marshal; diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index fa684bf4d090..9ff322e53f37 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -721,6 +721,9 @@ xprt_rdma_send_request(struct rpc_rqst *rqst, struct rpc_task *task) if (!xprt_connected(xprt)) goto drop_connection; + if (!xprt_request_get_cong(xprt, rqst)) + return -EBADSLT; + rc = rpcrdma_marshal_req(r_xprt, rqst); if (rc < 0) goto failed_marshal; diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index b8143eded4af..8831e84a058a 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -609,6 +609,10 @@ static int xs_udp_send_request(struct rpc_rqst *req, struct rpc_task *task) if (!xprt_bound(xprt)) return -ENOTCONN; + + if (!xprt_request_get_cong(xprt, req)) + return -EBADSLT; + req->rq_xtime = ktime_get(); status = xs_sendpages(transport->sock, xs_addr(xprt), xprt->addrlen, xdr, 0, true, &sent); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 26/34] SUNRPC: Improve latency for interactive tasks 2018-09-04 21:05 ` [PATCH v2 25/34] SUNRPC: Support for congestion control when queuing is enabled Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 27/34] SUNRPC: Allow calls to xprt_transmit() to drain the entire transmit queue Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs One of the intentions with the priority queues was to ensure that no single process can hog the transport. The field task->tk_owner therefore identifies the RPC call's origin, and is intended to allow the RPC layer to organise queues for fairness. This commit therefore modifies the transmit queue to group requests by task->tk_owner, and ensures that we round robin among those groups. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/backchannel_rqst.c | 1 + net/sunrpc/xprt.c | 23 +++++++++++++++++++++-- net/sunrpc/xprtrdma/backchannel.c | 1 + 4 files changed, 24 insertions(+), 2 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index b23c757bebfc..5dd64bdb28cd 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -89,6 +89,7 @@ struct rpc_rqst { }; struct list_head rq_xmit; /* Send queue */ + struct list_head rq_xmit2; /* Send queue */ void *rq_buffer; /* Call XDR encode buffer */ size_t rq_callsize; diff --git a/net/sunrpc/backchannel_rqst.c b/net/sunrpc/backchannel_rqst.c index 39b394b7dae3..e43dbab2e3f0 100644 --- a/net/sunrpc/backchannel_rqst.c +++ b/net/sunrpc/backchannel_rqst.c @@ -93,6 +93,7 @@ struct rpc_rqst *xprt_alloc_bc_req(struct rpc_xprt *xprt, gfp_t gfp_flags) req->rq_xprt = xprt; INIT_LIST_HEAD(&req->rq_recv); INIT_LIST_HEAD(&req->rq_xmit); + INIT_LIST_HEAD(&req->rq_xmit2); INIT_LIST_HEAD(&req->rq_bc_list); /* Preallocate one XDR receive buffer */ diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index a7a93d61567f..8a4c5260eecd 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1043,6 +1043,7 @@ xprt_request_enqueue_transmit(struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; + struct rpc_rqst *pos; spin_lock(&xprt->queue_lock); if (list_empty(&req->rq_xmit) && xprt_request_need_transmit(task) && @@ -1052,7 +1053,6 @@ xprt_request_enqueue_transmit(struct rpc_task *task) * to the head of the list to avoid starvation issues. */ if (req->rq_cong) { - struct rpc_rqst *pos; list_for_each_entry(pos, &xprt->xmit_queue, rq_xmit) { if (pos->rq_cong) continue; @@ -1060,6 +1060,13 @@ xprt_request_enqueue_transmit(struct rpc_task *task) list_add_tail(&req->rq_xmit, &pos->rq_xmit); goto out; } + } else { + list_for_each_entry(pos, &xprt->xmit_queue, rq_xmit) { + if (pos->rq_task->tk_owner != task->tk_owner) + continue; + list_add_tail(&req->rq_xmit2, &pos->rq_xmit2); + goto out; + } } list_add_tail(&req->rq_xmit, &xprt->xmit_queue); } @@ -1077,8 +1084,19 @@ xprt_request_enqueue_transmit(struct rpc_task *task) static void xprt_request_dequeue_transmit_locked(struct rpc_task *task) { + struct rpc_rqst *req = task->tk_rqstp; + clear_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); - list_del_init(&task->tk_rqstp->rq_xmit); + if (!list_empty(&req->rq_xmit)) { + list_del_init(&req->rq_xmit); + if (!list_empty(&req->rq_xmit2)) { + struct rpc_rqst *next = list_first_entry(&req->rq_xmit2, + struct rpc_rqst, rq_xmit2); + list_del_init(&req->rq_xmit2); + list_add_tail(&next->rq_xmit, &next->rq_xprt->xmit_queue); + } + } else if (!list_empty(&req->rq_xmit2)) + list_del_init(&req->rq_xmit2); } /** @@ -1376,6 +1394,7 @@ xprt_request_init(struct rpc_task *task) INIT_LIST_HEAD(&req->rq_recv); INIT_LIST_HEAD(&req->rq_xmit); + INIT_LIST_HEAD(&req->rq_xmit2); req->rq_timeout = task->tk_client->cl_timeout->to_initval; req->rq_task = task; req->rq_xprt = xprt; diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c index 14fc4596075e..92fffe765731 100644 --- a/net/sunrpc/xprtrdma/backchannel.c +++ b/net/sunrpc/xprtrdma/backchannel.c @@ -53,6 +53,7 @@ static int rpcrdma_bc_setup_reqs(struct rpcrdma_xprt *r_xprt, rqst->rq_xprt = xprt; INIT_LIST_HEAD(&rqst->rq_recv); INIT_LIST_HEAD(&rqst->rq_xmit); + INIT_LIST_HEAD(&rqst->rq_xmit2); INIT_LIST_HEAD(&rqst->rq_bc_list); __set_bit(RPC_BC_PA_IN_USE, &rqst->rq_bc_pa_state); spin_lock_bh(&xprt->bc_pa_lock); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 27/34] SUNRPC: Allow calls to xprt_transmit() to drain the entire transmit queue 2018-09-04 21:05 ` [PATCH v2 26/34] SUNRPC: Improve latency for interactive tasks Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 28/34] SUNRPC: Queue the request for transmission immediately after encoding Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Rather than forcing each and every RPC task to grab the socket write lock in order to send itself, we allow whichever task is holding the write lock to attempt to drain the entire transmit queue. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprt.c | 79 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 66 insertions(+), 13 deletions(-) diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 8a4c5260eecd..ba9af25d14de 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1139,15 +1139,20 @@ void xprt_end_transmit(struct rpc_task *task) } /** - * xprt_transmit - send an RPC request on a transport - * @task: controlling RPC task + * xprt_request_transmit - send an RPC request on a transport + * @req: pointer to request to transmit + * @snd_task: RPC task that owns the transport lock * - * We have to copy the iovec because sendmsg fiddles with its contents. + * This performs the transmission of a single request. + * Note that if the request is not the same as snd_task, then it + * does need to be pinned. + * Returns '0' on success. */ -void xprt_transmit(struct rpc_task *task) +static int +xprt_request_transmit(struct rpc_rqst *req, struct rpc_task *snd_task) { - struct rpc_rqst *req = task->tk_rqstp; - struct rpc_xprt *xprt = req->rq_xprt; + struct rpc_xprt *xprt = req->rq_xprt; + struct rpc_task *task = req->rq_task; unsigned int connect_cookie; int is_retrans = RPC_WAS_SENT(task); int status; @@ -1155,11 +1160,13 @@ void xprt_transmit(struct rpc_task *task) dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, req->rq_slen); if (!req->rq_bytes_sent) { - if (xprt_request_data_received(task)) + if (xprt_request_data_received(task)) { + status = 0; goto out_dequeue; + } /* Verify that our message lies in the RPCSEC_GSS window */ if (rpcauth_xmit_need_reencode(task)) { - task->tk_status = -EBADMSG; + status = -EBADMSG; goto out_dequeue; } } @@ -1167,12 +1174,10 @@ void xprt_transmit(struct rpc_task *task) req->rq_ntrans++; connect_cookie = xprt->connect_cookie; - status = xprt->ops->send_request(req, task); + status = xprt->ops->send_request(req, snd_task); trace_xprt_transmit(xprt, req->rq_xid, status); - if (status != 0) { - task->tk_status = status; - return; - } + if (status != 0) + return status; if (is_retrans) task->tk_client->cl_stats->rpcretrans++; @@ -1193,6 +1198,54 @@ void xprt_transmit(struct rpc_task *task) req->rq_connect_cookie = connect_cookie; out_dequeue: xprt_request_dequeue_transmit(task); + rpc_wake_up_queued_task_set_status(&xprt->sending, task, status); + return status; +} + +/** + * xprt_transmit - send an RPC request on a transport + * @task: controlling RPC task + * + * Attempts to drain the transmit queue. On exit, either the transport + * signalled an error that needs to be handled before transmission can + * resume, or @task finished transmitting, and detected that it already + * received a reply. + */ +void +xprt_transmit(struct rpc_task *task) +{ + struct rpc_rqst *next, *req = task->tk_rqstp; + struct rpc_xprt *xprt = req->rq_xprt; + LIST_HEAD(head); + int status; + + task->tk_status = -EAGAIN; + spin_lock(&xprt->queue_lock); + /* Avoid livelock by moving the xmit_queue contents to a private list */ + list_splice_init(&xprt->xmit_queue, &head); + while (!list_empty(&head)) { + next = list_first_entry(&head, struct rpc_rqst, rq_xmit); + xprt_pin_rqst(next); + spin_unlock(&xprt->queue_lock); + status = xprt_request_transmit(next, task); + if (status == -EBADMSG && next != req) + status = 0; + cond_resched(); + spin_lock(&xprt->queue_lock); + xprt_unpin_rqst(next); + if (status == 0) { + if (!xprt_request_data_received(task) || + test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) + continue; + } else if (!test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) + rpc_wake_up_queued_task(&xprt->pending, task); + else + task->tk_status = status; + /* On early exit, splice back the list contents */ + list_splice(&head, &xprt->xmit_queue); + break; + } + spin_unlock(&xprt->queue_lock); } static void xprt_add_backlog(struct rpc_xprt *xprt, struct rpc_task *task) -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 28/34] SUNRPC: Queue the request for transmission immediately after encoding 2018-09-04 21:05 ` [PATCH v2 27/34] SUNRPC: Allow calls to xprt_transmit() to drain the entire transmit queue Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Move up the call to xprt_request_enqueue_transmit() to call_encode() so that the queue order reflects the order in which slots were allocated. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/clnt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 52494baca7bc..4f8803413499 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1785,6 +1785,7 @@ call_encode(struct rpc_task *task) if (rpc_reply_expected(task)) xprt_request_enqueue_receive(task); set_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate); + xprt_request_enqueue_transmit(task); out: task->tk_action = call_bind; } @@ -1971,7 +1972,6 @@ call_transmit(struct rpc_task *task) dprint_status(task); task->tk_action = call_transmit_status; - xprt_request_enqueue_transmit(task); if (!test_bit(RPC_TASK_NEED_XMIT, &task->tk_runstate)) return; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue 2018-09-04 21:05 ` [PATCH v2 28/34] SUNRPC: Queue the request for transmission immediately after encoding Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 30/34] SUNRPC: Allow soft RPC calls to time out when waiting for the XPRT_LOCK Trond Myklebust 2018-09-06 14:17 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Schumaker, Anna 0 siblings, 2 replies; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs We no longer need priority semantics on the xprt->sending queue, because the order in which tasks are sent is now dictated by their position in the send queue. Note that the backlog queue remains a priority queue, meaning that slot resources are still managed in order of task priority. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprt.c | 29 ++++++----------------------- 1 file changed, 6 insertions(+), 23 deletions(-) diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index ba9af25d14de..ebe5235c484b 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -181,7 +181,6 @@ EXPORT_SYMBOL_GPL(xprt_load_transport); int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; - int priority; if (test_and_set_bit(XPRT_LOCKED, &xprt->state)) { if (task == xprt->snd_task) @@ -197,13 +196,7 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task) task->tk_pid, xprt); task->tk_timeout = 0; task->tk_status = -EAGAIN; - if (req == NULL) - priority = RPC_PRIORITY_LOW; - else if (!req->rq_ntrans) - priority = RPC_PRIORITY_NORMAL; - else - priority = RPC_PRIORITY_HIGH; - rpc_sleep_on_priority(&xprt->sending, task, NULL, priority); + rpc_sleep_on(&xprt->sending, task, NULL); return 0; } EXPORT_SYMBOL_GPL(xprt_reserve_xprt); @@ -231,7 +224,6 @@ static void xprt_clear_locked(struct rpc_xprt *xprt) int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) { struct rpc_rqst *req = task->tk_rqstp; - int priority; if (test_and_set_bit(XPRT_LOCKED, &xprt->state)) { if (task == xprt->snd_task) @@ -251,13 +243,7 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) dprintk("RPC: %5u failed to lock transport %p\n", task->tk_pid, xprt); task->tk_timeout = 0; task->tk_status = -EAGAIN; - if (req == NULL) - priority = RPC_PRIORITY_LOW; - else if (!req->rq_ntrans) - priority = RPC_PRIORITY_NORMAL; - else - priority = RPC_PRIORITY_HIGH; - rpc_sleep_on_priority(&xprt->sending, task, NULL, priority); + rpc_sleep_on(&xprt->sending, task, NULL); return 0; } EXPORT_SYMBOL_GPL(xprt_reserve_xprt_cong); @@ -285,8 +271,7 @@ static void __xprt_lock_write_next(struct rpc_xprt *xprt) if (test_and_set_bit(XPRT_LOCKED, &xprt->state)) return; - if (rpc_wake_up_first_on_wq(xprtiod_workqueue, &xprt->sending, - __xprt_lock_write_func, xprt)) + if (rpc_wake_up_first(&xprt->sending, __xprt_lock_write_func, xprt)) return; xprt_clear_locked(xprt); } @@ -297,8 +282,7 @@ static void __xprt_lock_write_next_cong(struct rpc_xprt *xprt) return; if (RPCXPRT_CONGESTED(xprt)) goto out_unlock; - if (rpc_wake_up_first_on_wq(xprtiod_workqueue, &xprt->sending, - __xprt_lock_write_func, xprt)) + if (rpc_wake_up_first(&xprt->sending, __xprt_lock_write_func, xprt)) return; out_unlock: xprt_clear_locked(xprt); @@ -495,8 +479,7 @@ void xprt_write_space(struct rpc_xprt *xprt) if (xprt->snd_task) { dprintk("RPC: write space: waking waiting task on " "xprt %p\n", xprt); - rpc_wake_up_queued_task_on_wq(xprtiod_workqueue, - &xprt->pending, xprt->snd_task); + rpc_wake_up_queued_task(&xprt->pending, xprt->snd_task); } spin_unlock_bh(&xprt->transport_lock); } @@ -1596,7 +1579,7 @@ static void xprt_init(struct rpc_xprt *xprt, struct net *net) rpc_init_wait_queue(&xprt->binding, "xprt_binding"); rpc_init_wait_queue(&xprt->pending, "xprt_pending"); - rpc_init_priority_wait_queue(&xprt->sending, "xprt_sending"); + rpc_init_wait_queue(&xprt->sending, "xprt_sending"); rpc_init_priority_wait_queue(&xprt->backlog, "xprt_backlog"); xprt_init_xid(xprt); -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 30/34] SUNRPC: Allow soft RPC calls to time out when waiting for the XPRT_LOCK 2018-09-04 21:05 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 31/34] SUNRPC: Turn off throttling of RPC slots for TCP sockets Trond Myklebust 2018-09-06 14:17 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Schumaker, Anna 1 sibling, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs This no longer causes them to lose their place in the transmission queue. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- net/sunrpc/xprt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index ebe5235c484b..bd818b97cb09 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -194,7 +194,7 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task) out_sleep: dprintk("RPC: %5u failed to lock transport %p\n", task->tk_pid, xprt); - task->tk_timeout = 0; + task->tk_timeout = RPC_IS_SOFT(task) ? req->rq_timeout : 0; task->tk_status = -EAGAIN; rpc_sleep_on(&xprt->sending, task, NULL); return 0; @@ -241,7 +241,7 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) xprt_clear_locked(xprt); out_sleep: dprintk("RPC: %5u failed to lock transport %p\n", task->tk_pid, xprt); - task->tk_timeout = 0; + task->tk_timeout = RPC_IS_SOFT(task) ? req->rq_timeout : 0; task->tk_status = -EAGAIN; rpc_sleep_on(&xprt->sending, task, NULL); return 0; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 31/34] SUNRPC: Turn off throttling of RPC slots for TCP sockets 2018-09-04 21:05 ` [PATCH v2 30/34] SUNRPC: Allow soft RPC calls to time out when waiting for the XPRT_LOCK Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 32/34] SUNRPC: Clean up transport write space handling Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs The theory was that we would need to grab the socket lock anyway, so we might as well use it to gate the allocation of RPC slots for a TCP socket. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 1 - net/sunrpc/xprt.c | 14 -------------- net/sunrpc/xprtsock.c | 2 +- 3 files changed, 1 insertion(+), 16 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 5dd64bdb28cd..d623bebab4f9 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -343,7 +343,6 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task); void xprt_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task); void xprt_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *req); -void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task); bool xprt_prepare_transmit(struct rpc_task *task); void xprt_request_enqueue_transmit(struct rpc_task *task); void xprt_request_enqueue_receive(struct rpc_task *task); diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index bd818b97cb09..4068d0ea3a21 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1325,20 +1325,6 @@ void xprt_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task) } EXPORT_SYMBOL_GPL(xprt_alloc_slot); -void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task) -{ - /* Note: grabbing the xprt_lock_write() ensures that we throttle - * new slot allocation if the transport is congested (i.e. when - * reconnecting a stream transport or when out of socket write - * buffer space). - */ - if (xprt_lock_write(xprt, task)) { - xprt_alloc_slot(xprt, task); - xprt_release_write(xprt, task); - } -} -EXPORT_SYMBOL_GPL(xprt_lock_and_alloc_slot); - void xprt_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *req) { spin_lock(&xprt->reserve_lock); diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 8831e84a058a..f54e8110f4c6 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -2809,7 +2809,7 @@ static const struct rpc_xprt_ops xs_udp_ops = { static const struct rpc_xprt_ops xs_tcp_ops = { .reserve_xprt = xprt_reserve_xprt, .release_xprt = xprt_release_xprt, - .alloc_slot = xprt_lock_and_alloc_slot, + .alloc_slot = xprt_alloc_slot, .free_slot = xprt_free_slot, .rpcbind = rpcb_getport_async, .set_port = xs_set_port, -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 32/34] SUNRPC: Clean up transport write space handling 2018-09-04 21:05 ` [PATCH v2 31/34] SUNRPC: Turn off throttling of RPC slots for TCP sockets Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 33/34] SUNRPC: Cleanup: remove the unused 'task' argument from the request_send() Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Treat socket write space handling in the same way we now treat transport congestion: by denying the XPRT_LOCK until the transport signals that it has free buffer space. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/svc_xprt.h | 1 - include/linux/sunrpc/xprt.h | 5 +- net/sunrpc/clnt.c | 11 +--- net/sunrpc/svc_xprt.c | 2 - net/sunrpc/xprt.c | 69 ++++++++++++++-------- net/sunrpc/xprtrdma/rpc_rdma.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 7 +-- net/sunrpc/xprtsock.c | 33 ++++------- 8 files changed, 62 insertions(+), 68 deletions(-) diff --git a/include/linux/sunrpc/svc_xprt.h b/include/linux/sunrpc/svc_xprt.h index c3d72066d4b1..6b7a86c4d6e6 100644 --- a/include/linux/sunrpc/svc_xprt.h +++ b/include/linux/sunrpc/svc_xprt.h @@ -84,7 +84,6 @@ struct svc_xprt { struct sockaddr_storage xpt_remote; /* remote peer's address */ size_t xpt_remotelen; /* length of address */ char xpt_remotebuf[INET6_ADDRSTRLEN + 10]; - struct rpc_wait_queue xpt_bc_pending; /* backchannel wait queue */ struct list_head xpt_users; /* callbacks on free */ struct net *xpt_net; diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index d623bebab4f9..ba4a9d83102a 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -386,8 +386,8 @@ int xprt_load_transport(const char *); void xprt_set_retrans_timeout_def(struct rpc_task *task); void xprt_set_retrans_timeout_rtt(struct rpc_task *task); void xprt_wake_pending_tasks(struct rpc_xprt *xprt, int status); -void xprt_wait_for_buffer_space(struct rpc_task *task, rpc_action action); -void xprt_write_space(struct rpc_xprt *xprt); +void xprt_wait_for_buffer_space(struct rpc_xprt *xprt); +bool xprt_write_space(struct rpc_xprt *xprt); void xprt_adjust_cwnd(struct rpc_xprt *xprt, struct rpc_task *task, int result); struct rpc_rqst * xprt_lookup_rqst(struct rpc_xprt *xprt, __be32 xid); void xprt_update_rtt(struct rpc_task *task); @@ -414,6 +414,7 @@ void xprt_unlock_connect(struct rpc_xprt *, void *); #define XPRT_BINDING (5) #define XPRT_CLOSING (6) #define XPRT_CONGESTED (9) +#define XPRT_WRITE_SPACE (10) static inline void xprt_set_connected(struct rpc_xprt *xprt) { diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 4f8803413499..fb19c8a85e68 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1978,6 +1978,7 @@ call_transmit(struct rpc_task *task) if (!xprt_prepare_transmit(task)) return; xprt_transmit(task); + xprt_end_transmit(task); } /* @@ -1993,7 +1994,6 @@ call_transmit_status(struct rpc_task *task) * test first. */ if (task->tk_status == 0) { - xprt_end_transmit(task); xprt_request_wait_receive(task); return; } @@ -2001,15 +2001,12 @@ call_transmit_status(struct rpc_task *task) switch (task->tk_status) { default: dprint_status(task); - xprt_end_transmit(task); break; case -EBADSLT: - xprt_end_transmit(task); task->tk_action = call_transmit; task->tk_status = 0; break; case -EBADMSG: - xprt_end_transmit(task); task->tk_action = call_encode; break; /* @@ -2032,7 +2029,6 @@ call_transmit_status(struct rpc_task *task) case -ENETUNREACH: case -EPERM: if (RPC_IS_SOFTCONN(task)) { - xprt_end_transmit(task); if (!task->tk_msg.rpc_proc->p_proc) trace_xprt_ping(task->tk_xprt, task->tk_status); @@ -2076,9 +2072,6 @@ call_bc_transmit(struct rpc_task *task) xprt_transmit(task); - if (task->tk_status == -EAGAIN) - goto out_nospace; - xprt_end_transmit(task); dprint_status(task); switch (task->tk_status) { @@ -2094,6 +2087,8 @@ call_bc_transmit(struct rpc_task *task) case -ENOTCONN: case -EPIPE: break; + case -EAGAIN: + goto out_nospace; case -ETIMEDOUT: /* * Problem reaching the server. Disconnect and let the diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index 5185efb9027b..87533fbb96cf 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -171,7 +171,6 @@ void svc_xprt_init(struct net *net, struct svc_xprt_class *xcl, mutex_init(&xprt->xpt_mutex); spin_lock_init(&xprt->xpt_lock); set_bit(XPT_BUSY, &xprt->xpt_flags); - rpc_init_wait_queue(&xprt->xpt_bc_pending, "xpt_bc_pending"); xprt->xpt_net = get_net(net); strcpy(xprt->xpt_remotebuf, "uninitialized"); } @@ -895,7 +894,6 @@ int svc_send(struct svc_rqst *rqstp) else len = xprt->xpt_ops->xpo_sendto(rqstp); mutex_unlock(&xprt->xpt_mutex); - rpc_wake_up(&xprt->xpt_bc_pending); trace_svc_send(rqstp, len); svc_xprt_release(rqstp); diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 4068d0ea3a21..baa454b0d855 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -169,6 +169,17 @@ int xprt_load_transport(const char *transport_name) } EXPORT_SYMBOL_GPL(xprt_load_transport); +static void xprt_clear_locked(struct rpc_xprt *xprt) +{ + xprt->snd_task = NULL; + if (!test_bit(XPRT_CLOSE_WAIT, &xprt->state)) { + smp_mb__before_atomic(); + clear_bit(XPRT_LOCKED, &xprt->state); + smp_mb__after_atomic(); + } else + queue_work(xprtiod_workqueue, &xprt->task_cleanup); +} + /** * xprt_reserve_xprt - serialize write access to transports * @task: task that is requesting access to the transport @@ -187,10 +198,14 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task) return 1; goto out_sleep; } + if (test_bit(XPRT_WRITE_SPACE, &xprt->state)) + goto out_unlock; xprt->snd_task = task; return 1; +out_unlock: + xprt_clear_locked(xprt); out_sleep: dprintk("RPC: %5u failed to lock transport %p\n", task->tk_pid, xprt); @@ -201,17 +216,6 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task) } EXPORT_SYMBOL_GPL(xprt_reserve_xprt); -static void xprt_clear_locked(struct rpc_xprt *xprt) -{ - xprt->snd_task = NULL; - if (!test_bit(XPRT_CLOSE_WAIT, &xprt->state)) { - smp_mb__before_atomic(); - clear_bit(XPRT_LOCKED, &xprt->state); - smp_mb__after_atomic(); - } else - queue_work(xprtiod_workqueue, &xprt->task_cleanup); -} - /* * xprt_reserve_xprt_cong - serialize write access to transports * @task: task that is requesting access to the transport @@ -234,10 +238,13 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task) xprt->snd_task = task; return 1; } + if (test_bit(XPRT_WRITE_SPACE, &xprt->state)) + goto out_unlock; if (!RPCXPRT_CONGESTED(xprt)) { xprt->snd_task = task; return 1; } +out_unlock: xprt_clear_locked(xprt); out_sleep: dprintk("RPC: %5u failed to lock transport %p\n", task->tk_pid, xprt); @@ -270,9 +277,11 @@ static void __xprt_lock_write_next(struct rpc_xprt *xprt) { if (test_and_set_bit(XPRT_LOCKED, &xprt->state)) return; - + if (test_bit(XPRT_WRITE_SPACE, &xprt->state)) + goto out_unlock; if (rpc_wake_up_first(&xprt->sending, __xprt_lock_write_func, xprt)) return; +out_unlock: xprt_clear_locked(xprt); } @@ -280,6 +289,8 @@ static void __xprt_lock_write_next_cong(struct rpc_xprt *xprt) { if (test_and_set_bit(XPRT_LOCKED, &xprt->state)) return; + if (test_bit(XPRT_WRITE_SPACE, &xprt->state)) + goto out_unlock; if (RPCXPRT_CONGESTED(xprt)) goto out_unlock; if (rpc_wake_up_first(&xprt->sending, __xprt_lock_write_func, xprt)) @@ -450,38 +461,43 @@ EXPORT_SYMBOL_GPL(xprt_wake_pending_tasks); /** * xprt_wait_for_buffer_space - wait for transport output buffer to clear - * @task: task to be put to sleep - * @action: function pointer to be executed after wait + * @xprt: transport * * Note that we only set the timer for the case of RPC_IS_SOFT(), since * we don't in general want to force a socket disconnection due to * an incomplete RPC call transmission. */ -void xprt_wait_for_buffer_space(struct rpc_task *task, rpc_action action) +void xprt_wait_for_buffer_space(struct rpc_xprt *xprt) { - struct rpc_rqst *req = task->tk_rqstp; - struct rpc_xprt *xprt = req->rq_xprt; - - task->tk_timeout = RPC_IS_SOFT(task) ? req->rq_timeout : 0; - rpc_sleep_on(&xprt->pending, task, action); + set_bit(XPRT_WRITE_SPACE, &xprt->state); } EXPORT_SYMBOL_GPL(xprt_wait_for_buffer_space); +static bool +xprt_clear_write_space_locked(struct rpc_xprt *xprt) +{ + if (test_and_clear_bit(XPRT_WRITE_SPACE, &xprt->state)) { + __xprt_lock_write_next(xprt); + dprintk("RPC: write space: waking waiting task on " + "xprt %p\n", xprt); + return true; + } + return false; +} + /** * xprt_write_space - wake the task waiting for transport output buffer space * @xprt: transport with waiting tasks * * Can be called in a soft IRQ context, so xprt_write_space never sleeps. */ -void xprt_write_space(struct rpc_xprt *xprt) +bool xprt_write_space(struct rpc_xprt *xprt) { + bool ret; spin_lock_bh(&xprt->transport_lock); - if (xprt->snd_task) { - dprintk("RPC: write space: waking waiting task on " - "xprt %p\n", xprt); - rpc_wake_up_queued_task(&xprt->pending, xprt->snd_task); - } + ret = xprt_clear_write_space_locked(xprt); spin_unlock_bh(&xprt->transport_lock); + return ret; } EXPORT_SYMBOL_GPL(xprt_write_space); @@ -592,6 +608,7 @@ void xprt_disconnect_done(struct rpc_xprt *xprt) dprintk("RPC: disconnected transport %p\n", xprt); spin_lock_bh(&xprt->transport_lock); xprt_clear_connected(xprt); + xprt_clear_write_space_locked(xprt); xprt_wake_pending_tasks(xprt, -EAGAIN); spin_unlock_bh(&xprt->transport_lock); } diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 0020dc401215..53fa95d60015 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -866,7 +866,7 @@ rpcrdma_marshal_req(struct rpcrdma_xprt *r_xprt, struct rpc_rqst *rqst) out_err: switch (ret) { case -EAGAIN: - xprt_wait_for_buffer_space(rqst->rq_task, NULL); + xprt_wait_for_buffer_space(rqst->rq_xprt); break; case -ENOBUFS: break; diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index d1618c70edb4..35a8c3aab302 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -224,12 +224,7 @@ xprt_rdma_bc_send_request(struct rpc_rqst *rqst, struct rpc_task *task) dprintk("svcrdma: sending bc call with xid: %08x\n", be32_to_cpu(rqst->rq_xid)); - if (!mutex_trylock(&sxprt->xpt_mutex)) { - rpc_sleep_on(&sxprt->xpt_bc_pending, task, NULL); - if (!mutex_trylock(&sxprt->xpt_mutex)) - return -EAGAIN; - rpc_wake_up_queued_task(&sxprt->xpt_bc_pending, task); - } + mutex_lock(&sxprt->xpt_mutex); ret = -ENOTCONN; rdma = container_of(sxprt, struct svcxprt_rdma, sc_xprt); diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index f54e8110f4c6..ef8d0e81cbda 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -440,20 +440,12 @@ static int xs_sendpages(struct socket *sock, struct sockaddr *addr, int addrlen, return err; } -static void xs_nospace_callback(struct rpc_task *task) -{ - struct sock_xprt *transport = container_of(task->tk_rqstp->rq_xprt, struct sock_xprt, xprt); - - transport->inet->sk_write_pending--; -} - /** - * xs_nospace - place task on wait queue if transmit was incomplete + * xs_nospace - handle transmit was incomplete * @req: pointer to RPC request - * @task: task to put to sleep * */ -static int xs_nospace(struct rpc_rqst *req, struct rpc_task *task) +static int xs_nospace(struct rpc_rqst *req) { struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); @@ -461,7 +453,8 @@ static int xs_nospace(struct rpc_rqst *req, struct rpc_task *task) int ret = -EAGAIN; dprintk("RPC: %5u xmit incomplete (%u left of %u)\n", - task->tk_pid, req->rq_slen - transport->xmit.offset, + req->rq_task->tk_pid, + req->rq_slen - transport->xmit.offset, req->rq_slen); /* Protect against races with write_space */ @@ -471,7 +464,7 @@ static int xs_nospace(struct rpc_rqst *req, struct rpc_task *task) if (xprt_connected(xprt)) { /* wait for more buffer space */ sk->sk_write_pending++; - xprt_wait_for_buffer_space(task, xs_nospace_callback); + xprt_wait_for_buffer_space(xprt); } else ret = -ENOTCONN; @@ -569,7 +562,7 @@ static int xs_local_send_request(struct rpc_rqst *req, struct rpc_task *task) case -ENOBUFS: break; case -EAGAIN: - status = xs_nospace(req, task); + status = xs_nospace(req); break; default: dprintk("RPC: sendmsg returned unrecognized error %d\n", @@ -642,7 +635,7 @@ static int xs_udp_send_request(struct rpc_rqst *req, struct rpc_task *task) /* Should we call xs_close() here? */ break; case -EAGAIN: - status = xs_nospace(req, task); + status = xs_nospace(req); break; case -ENETUNREACH: case -ENOBUFS: @@ -765,7 +758,7 @@ static int xs_tcp_send_request(struct rpc_rqst *req, struct rpc_task *task) /* Should we call xs_close() here? */ break; case -EAGAIN: - status = xs_nospace(req, task); + status = xs_nospace(req); break; case -ECONNRESET: case -ECONNREFUSED: @@ -1672,7 +1665,8 @@ static void xs_write_space(struct sock *sk) if (!wq || test_and_clear_bit(SOCKWQ_ASYNC_NOSPACE, &wq->flags) == 0) goto out; - xprt_write_space(xprt); + if (xprt_write_space(xprt)) + sk->sk_write_pending--; out: rcu_read_unlock(); } @@ -2725,12 +2719,7 @@ static int bc_send_request(struct rpc_rqst *req, struct rpc_task *task) * Grab the mutex to serialize data as the connection is shared * with the fore channel */ - if (!mutex_trylock(&xprt->xpt_mutex)) { - rpc_sleep_on(&xprt->xpt_bc_pending, task, NULL); - if (!mutex_trylock(&xprt->xpt_mutex)) - return -EAGAIN; - rpc_wake_up_queued_task(&xprt->xpt_bc_pending, task); - } + mutex_lock(&xprt->xpt_mutex); if (test_bit(XPT_DEAD, &xprt->xpt_flags)) len = -ENOTCONN; else -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 33/34] SUNRPC: Cleanup: remove the unused 'task' argument from the request_send() 2018-09-04 21:05 ` [PATCH v2 32/34] SUNRPC: Clean up transport write space handling Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 34/34] SUNRPC: Queue fairness for all Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/xprt.h | 2 +- net/sunrpc/xprt.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 2 +- net/sunrpc/xprtrdma/transport.c | 4 ++-- net/sunrpc/xprtsock.c | 11 ++++------- 5 files changed, 9 insertions(+), 12 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index ba4a9d83102a..0120e53b4ce9 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -141,7 +141,7 @@ struct rpc_xprt_ops { void (*connect)(struct rpc_xprt *xprt, struct rpc_task *task); int (*buf_alloc)(struct rpc_task *task); void (*buf_free)(struct rpc_task *task); - int (*send_request)(struct rpc_rqst *req, struct rpc_task *task); + int (*send_request)(struct rpc_rqst *req); void (*set_retrans_timeout)(struct rpc_task *task); void (*timer)(struct rpc_xprt *xprt, struct rpc_task *task); void (*release_request)(struct rpc_task *task); diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index baa454b0d855..147921f60b5a 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1174,7 +1174,7 @@ xprt_request_transmit(struct rpc_rqst *req, struct rpc_task *snd_task) req->rq_ntrans++; connect_cookie = xprt->connect_cookie; - status = xprt->ops->send_request(req, snd_task); + status = xprt->ops->send_request(req); trace_xprt_transmit(xprt, req->rq_xid, status); if (status != 0) return status; diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index 35a8c3aab302..992312504cfd 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -215,7 +215,7 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) * connection. */ static int -xprt_rdma_bc_send_request(struct rpc_rqst *rqst, struct rpc_task *task) +xprt_rdma_bc_send_request(struct rpc_rqst *rqst) { struct svc_xprt *sxprt = rqst->rq_xprt->bc_xprt; struct svcxprt_rdma *rdma; diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 9ff322e53f37..a5a6a4a353f2 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -693,7 +693,7 @@ xprt_rdma_free(struct rpc_task *task) /** * xprt_rdma_send_request - marshal and send an RPC request - * @task: RPC task with an RPC message in rq_snd_buf + * @rqst: RPC message in rq_snd_buf * * Caller holds the transport's write lock. * @@ -706,7 +706,7 @@ xprt_rdma_free(struct rpc_task *task) * sent. Do not try to send this message again. */ static int -xprt_rdma_send_request(struct rpc_rqst *rqst, struct rpc_task *task) +xprt_rdma_send_request(struct rpc_rqst *rqst) { struct rpc_xprt *xprt = rqst->rq_xprt; struct rpcrdma_req *req = rpcr_to_rdmar(rqst); diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index ef8d0e81cbda..f16406228ead 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -507,7 +507,6 @@ static inline void xs_encode_stream_record_marker(struct xdr_buf *buf) /** * xs_local_send_request - write an RPC request to an AF_LOCAL socket * @req: pointer to RPC request - * @task: RPC task that manages the state of an RPC request * * Return values: * 0: The request has been sent @@ -516,7 +515,7 @@ static inline void xs_encode_stream_record_marker(struct xdr_buf *buf) * ENOTCONN: Caller needs to invoke connect logic then call again * other: Some other error occured, the request was not sent */ -static int xs_local_send_request(struct rpc_rqst *req, struct rpc_task *task) +static int xs_local_send_request(struct rpc_rqst *req) { struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = @@ -579,7 +578,6 @@ static int xs_local_send_request(struct rpc_rqst *req, struct rpc_task *task) /** * xs_udp_send_request - write an RPC request to a UDP socket * @req: pointer to RPC request - * @task: address of RPC task that manages the state of an RPC request * * Return values: * 0: The request has been sent @@ -588,7 +586,7 @@ static int xs_local_send_request(struct rpc_rqst *req, struct rpc_task *task) * ENOTCONN: Caller needs to invoke connect logic then call again * other: Some other error occurred, the request was not sent */ -static int xs_udp_send_request(struct rpc_rqst *req, struct rpc_task *task) +static int xs_udp_send_request(struct rpc_rqst *req) { struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); @@ -656,7 +654,6 @@ static int xs_udp_send_request(struct rpc_rqst *req, struct rpc_task *task) /** * xs_tcp_send_request - write an RPC request to a TCP socket * @req: pointer to RPC request - * @task: address of RPC task that manages the state of an RPC request * * Return values: * 0: The request has been sent @@ -668,7 +665,7 @@ static int xs_udp_send_request(struct rpc_rqst *req, struct rpc_task *task) * XXX: In the case of soft timeouts, should we eventually give up * if sendmsg is not able to make progress? */ -static int xs_tcp_send_request(struct rpc_rqst *req, struct rpc_task *task) +static int xs_tcp_send_request(struct rpc_rqst *req) { struct rpc_xprt *xprt = req->rq_xprt; struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt); @@ -2704,7 +2701,7 @@ static int bc_sendto(struct rpc_rqst *req) /* * The send routine. Borrows from svc_send */ -static int bc_send_request(struct rpc_rqst *req, struct rpc_task *task) +static int bc_send_request(struct rpc_rqst *req) { struct svc_xprt *xprt; int len; -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 34/34] SUNRPC: Queue fairness for all. 2018-09-04 21:05 ` [PATCH v2 33/34] SUNRPC: Cleanup: remove the unused 'task' argument from the request_send() Trond Myklebust @ 2018-09-04 21:05 ` Trond Myklebust 0 siblings, 0 replies; 45+ messages in thread From: Trond Myklebust @ 2018-09-04 21:05 UTC (permalink / raw) To: linux-nfs Fix up the priority queue to not batch by owner, but by queue, so that we allow '1 << priority' elements to be dequeued before switching to the next priority queue. The owner field is still used to wake up requests in round robin order by owner to avoid single processes hogging the RPC layer by loading the queues. We extend this property to non-priority queues as well. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> --- include/linux/sunrpc/sched.h | 2 - net/sunrpc/sched.c | 126 ++++++++++++++++------------------- 2 files changed, 57 insertions(+), 71 deletions(-) diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index c5bc779feb00..869e2ee787fa 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -188,7 +188,6 @@ struct rpc_timer { struct rpc_wait_queue { spinlock_t lock; struct list_head tasks[RPC_NR_PRIORITY]; /* task queue for each priority level */ - pid_t owner; /* process id of last task serviced */ unsigned char maxpriority; /* maximum priority (0 if queue is not a priority queue) */ unsigned char priority; /* current priority */ unsigned char nr; /* # tasks remaining for cookie */ @@ -204,7 +203,6 @@ struct rpc_wait_queue { * from a single cookie. The aim is to improve * performance of NFS operations such as read/write. */ -#define RPC_BATCH_COUNT 16 #define RPC_IS_PRIORITY(q) ((q)->maxpriority > 0) /* diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c index 104c056daf83..1120857eb1df 100644 --- a/net/sunrpc/sched.c +++ b/net/sunrpc/sched.c @@ -99,64 +99,77 @@ __rpc_add_timer(struct rpc_wait_queue *queue, struct rpc_task *task) list_add(&task->u.tk_wait.timer_list, &queue->timer_list.list); } -static void rpc_rotate_queue_owner(struct rpc_wait_queue *queue) -{ - struct list_head *q = &queue->tasks[queue->priority]; - struct rpc_task *task; - - if (!list_empty(q)) { - task = list_first_entry(q, struct rpc_task, u.tk_wait.list); - if (task->tk_owner == queue->owner) - list_move_tail(&task->u.tk_wait.list, q); - } -} - static void rpc_set_waitqueue_priority(struct rpc_wait_queue *queue, int priority) { if (queue->priority != priority) { - /* Fairness: rotate the list when changing priority */ - rpc_rotate_queue_owner(queue); queue->priority = priority; + queue->nr = 1U << priority; } } -static void rpc_set_waitqueue_owner(struct rpc_wait_queue *queue, pid_t pid) -{ - queue->owner = pid; - queue->nr = RPC_BATCH_COUNT; -} - static void rpc_reset_waitqueue_priority(struct rpc_wait_queue *queue) { rpc_set_waitqueue_priority(queue, queue->maxpriority); - rpc_set_waitqueue_owner(queue, 0); } /* - * Add new request to a priority queue. + * Add a request to a queue list */ -static void __rpc_add_wait_queue_priority(struct rpc_wait_queue *queue, - struct rpc_task *task, - unsigned char queue_priority) +static void +__rpc_list_enqueue_task(struct list_head *q, struct rpc_task *task) { - struct list_head *q; struct rpc_task *t; - - INIT_LIST_HEAD(&task->u.tk_wait.links); - if (unlikely(queue_priority > queue->maxpriority)) - queue_priority = queue->maxpriority; - if (queue_priority > queue->priority) - rpc_set_waitqueue_priority(queue, queue_priority); - q = &queue->tasks[queue_priority]; list_for_each_entry(t, q, u.tk_wait.list) { if (t->tk_owner == task->tk_owner) { - list_add_tail(&task->u.tk_wait.list, &t->u.tk_wait.links); + list_add_tail(&task->u.tk_wait.links, + &t->u.tk_wait.links); + /* Cache the queue head in task->u.tk_wait.list */ + task->u.tk_wait.list.next = q; + task->u.tk_wait.list.prev = NULL; return; } } + INIT_LIST_HEAD(&task->u.tk_wait.links); list_add_tail(&task->u.tk_wait.list, q); } +/* + * Remove request from a queue list + */ +static void +__rpc_list_dequeue_task(struct rpc_task *task) +{ + struct list_head *q; + struct rpc_task *t; + + if (task->u.tk_wait.list.prev == NULL) { + list_del(&task->u.tk_wait.links); + return; + } + if (!list_empty(&task->u.tk_wait.links)) { + t = list_first_entry(&task->u.tk_wait.links, + struct rpc_task, + u.tk_wait.links); + /* Assume __rpc_list_enqueue_task() cached the queue head */ + q = t->u.tk_wait.list.next; + list_add_tail(&t->u.tk_wait.list, q); + list_del(&task->u.tk_wait.links); + } + list_del(&task->u.tk_wait.list); +} + +/* + * Add new request to a priority queue. + */ +static void __rpc_add_wait_queue_priority(struct rpc_wait_queue *queue, + struct rpc_task *task, + unsigned char queue_priority) +{ + if (unlikely(queue_priority > queue->maxpriority)) + queue_priority = queue->maxpriority; + __rpc_list_enqueue_task(&queue->tasks[queue_priority], task); +} + /* * Add new request to wait queue. * @@ -175,10 +188,11 @@ static void __rpc_add_wait_queue(struct rpc_wait_queue *queue, if (RPC_IS_PRIORITY(queue)) __rpc_add_wait_queue_priority(queue, task, queue_priority); - else if (RPC_IS_SWAPPER(task)) + else if (RPC_IS_SWAPPER(task)) { list_add(&task->u.tk_wait.list, &queue->tasks[0]); - else - list_add_tail(&task->u.tk_wait.list, &queue->tasks[0]); + INIT_LIST_HEAD(&task->u.tk_wait.links); + } else + __rpc_list_enqueue_task(&queue->tasks[0], task); task->tk_waitqueue = queue; queue->qlen++; /* barrier matches the read in rpc_wake_up_task_queue_locked() */ @@ -189,20 +203,6 @@ static void __rpc_add_wait_queue(struct rpc_wait_queue *queue, task->tk_pid, queue, rpc_qname(queue)); } -/* - * Remove request from a priority queue. - */ -static void __rpc_remove_wait_queue_priority(struct rpc_task *task) -{ - struct rpc_task *t; - - if (!list_empty(&task->u.tk_wait.links)) { - t = list_entry(task->u.tk_wait.links.next, struct rpc_task, u.tk_wait.list); - list_move(&t->u.tk_wait.list, &task->u.tk_wait.list); - list_splice_init(&task->u.tk_wait.links, &t->u.tk_wait.links); - } -} - /* * Remove request from queue. * Note: must be called with spin lock held. @@ -210,9 +210,7 @@ static void __rpc_remove_wait_queue_priority(struct rpc_task *task) static void __rpc_remove_wait_queue(struct rpc_wait_queue *queue, struct rpc_task *task) { __rpc_disable_timer(queue, task); - if (RPC_IS_PRIORITY(queue)) - __rpc_remove_wait_queue_priority(task); - list_del(&task->u.tk_wait.list); + __rpc_list_dequeue_task(task); queue->qlen--; dprintk("RPC: %5u removed from queue %p \"%s\"\n", task->tk_pid, queue, rpc_qname(queue)); @@ -536,20 +534,12 @@ static struct rpc_task *__rpc_find_next_queued_priority(struct rpc_wait_queue *q struct rpc_task *task; /* - * Service a batch of tasks from a single owner. + * Service a batch of tasks from a single queue. */ q = &queue->tasks[queue->priority]; - if (!list_empty(q)) { - task = list_entry(q->next, struct rpc_task, u.tk_wait.list); - if (queue->owner == task->tk_owner) { - if (--queue->nr) - goto out; - list_move_tail(&task->u.tk_wait.list, q); - } - /* - * Check if we need to switch queues. - */ - goto new_owner; + if (!list_empty(q) && --queue->nr) { + task = list_first_entry(q, struct rpc_task, u.tk_wait.list); + goto out; } /* @@ -561,7 +551,7 @@ static struct rpc_task *__rpc_find_next_queued_priority(struct rpc_wait_queue *q else q = q - 1; if (!list_empty(q)) { - task = list_entry(q->next, struct rpc_task, u.tk_wait.list); + task = list_first_entry(q, struct rpc_task, u.tk_wait.list); goto new_queue; } } while (q != &queue->tasks[queue->priority]); @@ -571,8 +561,6 @@ static struct rpc_task *__rpc_find_next_queued_priority(struct rpc_wait_queue *q new_queue: rpc_set_waitqueue_priority(queue, (unsigned int)(q - &queue->tasks[0])); -new_owner: - rpc_set_waitqueue_owner(queue, task->tk_owner); out: return task; } -- 2.17.1 ^ permalink raw reply related [flat|nested] 45+ messages in thread
* Re: [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue 2018-09-04 21:05 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 30/34] SUNRPC: Allow soft RPC calls to time out when waiting for the XPRT_LOCK Trond Myklebust @ 2018-09-06 14:17 ` Schumaker, Anna 2018-09-06 14:23 ` Schumaker, Anna 2018-09-06 14:23 ` Schumaker, Anna 1 sibling, 2 replies; 45+ messages in thread From: Schumaker, Anna @ 2018-09-06 14:17 UTC (permalink / raw) To: linux-nfs, trondmy SGkgVHJvbmQsDQoNCk9uIFR1ZSwgMjAxOC0wOS0wNCBhdCAxNzowNSAtMDQwMCwgVHJvbmQgTXlr bGVidXN0IHdyb3RlOg0KPiBXZSBubyBsb25nZXIgbmVlZCBwcmlvcml0eSBzZW1hbnRpY3Mgb24g dGhlIHhwcnQtPnNlbmRpbmcgcXVldWUsIGJlY2F1c2UNCj4gdGhlIG9yZGVyIGluIHdoaWNoIHRh c2tzIGFyZSBzZW50IGlzIG5vdyBkaWN0YXRlZCBieSB0aGVpciBwb3NpdGlvbiBpbg0KPiB0aGUg c2VuZCBxdWV1ZS4NCj4gTm90ZSB0aGF0IHRoZSBiYWNrbG9nIHF1ZXVlIHJlbWFpbnMgYSBwcmlv cml0eSBxdWV1ZSwgbWVhbmluZyB0aGF0DQo+IHNsb3QgcmVzb3VyY2VzIGFyZSBzdGlsbCBtYW5h Z2VkIGluIG9yZGVyIG9mIHRhc2sgcHJpb3JpdHkuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBUcm9u ZCBNeWtsZWJ1c3QgPHRyb25kLm15a2xlYnVzdEBoYW1tZXJzcGFjZS5jb20+DQo+IC0tLQ0KPiAg bmV0L3N1bnJwYy94cHJ0LmMgfCAyOSArKysrKystLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiAg MSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKSwgMjMgZGVsZXRpb25zKC0pDQo+IA0KPiBk aWZmIC0tZ2l0IGEvbmV0L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiBpbmRl eCBiYTlhZjI1ZDE0ZGUuLmViZTUyMzVjNDg0YiAxMDA2NDQNCj4gLS0tIGEvbmV0L3N1bnJwYy94 cHJ0LmMNCj4gKysrIGIvbmV0L3N1bnJwYy94cHJ0LmMNCj4gQEAgLTE4MSw3ICsxODEsNiBAQCBF WFBPUlRfU1lNQk9MX0dQTCh4cHJ0X2xvYWRfdHJhbnNwb3J0KTsNCj4gIGludCB4cHJ0X3Jlc2Vy dmVfeHBydChzdHJ1Y3QgcnBjX3hwcnQgKnhwcnQsIHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4g IHsNCj4gIAlzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSA9IHRhc2stPnRrX3Jxc3RwOw0KDQpBcyBvZiB0 aGlzIHBhdGNoLCByZXEgaXMgbm8gbG9uZ2VyIHVzZWQgZWl0aGVyLiAgQ2FuIGl0IGJlIHJlbW92 ZWQ/DQoNClRoYW5rcywNCkFubmENCg0KPiAtCWludCBwcmlvcml0eTsNCj4gIA0KPiAgCWlmICh0 ZXN0X2FuZF9zZXRfYml0KFhQUlRfTE9DS0VELCAmeHBydC0+c3RhdGUpKSB7DQo+ICAJCWlmICh0 YXNrID09IHhwcnQtPnNuZF90YXNrKQ0KPiBAQCAtMTk3LDEzICsxOTYsNyBAQCBpbnQgeHBydF9y ZXNlcnZlX3hwcnQoc3RydWN0IHJwY194cHJ0ICp4cHJ0LCBzdHJ1Y3QNCj4gcnBjX3Rhc2sgKnRh c2spDQo+ICAJCQl0YXNrLT50a19waWQsIHhwcnQpOw0KPiAgCXRhc2stPnRrX3RpbWVvdXQgPSAw Ow0KPiAgCXRhc2stPnRrX3N0YXR1cyA9IC1FQUdBSU47DQo+IC0JaWYgKHJlcSA9PSBOVUxMKQ0K PiAtCQlwcmlvcml0eSA9IFJQQ19QUklPUklUWV9MT1c7DQo+IC0JZWxzZSBpZiAoIXJlcS0+cnFf bnRyYW5zKQ0KPiAtCQlwcmlvcml0eSA9IFJQQ19QUklPUklUWV9OT1JNQUw7DQo+IC0JZWxzZQ0K PiAtCQlwcmlvcml0eSA9IFJQQ19QUklPUklUWV9ISUdIOw0KPiAtCXJwY19zbGVlcF9vbl9wcmlv cml0eSgmeHBydC0+c2VuZGluZywgdGFzaywgTlVMTCwgcHJpb3JpdHkpOw0KPiArCXJwY19zbGVl cF9vbigmeHBydC0+c2VuZGluZywgdGFzaywgTlVMTCk7DQo+ICAJcmV0dXJuIDA7DQo+ICB9DQo+ ICBFWFBPUlRfU1lNQk9MX0dQTCh4cHJ0X3Jlc2VydmVfeHBydCk7DQo+IEBAIC0yMzEsNyArMjI0 LDYgQEAgc3RhdGljIHZvaWQgeHBydF9jbGVhcl9sb2NrZWQoc3RydWN0IHJwY194cHJ0ICp4cHJ0 KQ0KPiAgaW50IHhwcnRfcmVzZXJ2ZV94cHJ0X2Nvbmcoc3RydWN0IHJwY194cHJ0ICp4cHJ0LCBz dHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ICB7DQo+ICAJc3RydWN0IHJwY19ycXN0ICpyZXEgPSB0 YXNrLT50a19ycXN0cDsNCj4gLQlpbnQgcHJpb3JpdHk7DQo+ICANCj4gIAlpZiAodGVzdF9hbmRf c2V0X2JpdChYUFJUX0xPQ0tFRCwgJnhwcnQtPnN0YXRlKSkgew0KPiAgCQlpZiAodGFzayA9PSB4 cHJ0LT5zbmRfdGFzaykNCj4gQEAgLTI1MSwxMyArMjQzLDcgQEAgaW50IHhwcnRfcmVzZXJ2ZV94 cHJ0X2Nvbmcoc3RydWN0IHJwY194cHJ0ICp4cHJ0LCBzdHJ1Y3QNCj4gcnBjX3Rhc2sgKnRhc2sp DQo+ICAJZHByaW50aygiUlBDOiAlNXUgZmFpbGVkIHRvIGxvY2sgdHJhbnNwb3J0ICVwXG4iLCB0 YXNrLT50a19waWQsIHhwcnQpOw0KPiAgCXRhc2stPnRrX3RpbWVvdXQgPSAwOw0KPiAgCXRhc2st PnRrX3N0YXR1cyA9IC1FQUdBSU47DQo+IC0JaWYgKHJlcSA9PSBOVUxMKQ0KPiAtCQlwcmlvcml0 eSA9IFJQQ19QUklPUklUWV9MT1c7DQo+IC0JZWxzZSBpZiAoIXJlcS0+cnFfbnRyYW5zKQ0KPiAt CQlwcmlvcml0eSA9IFJQQ19QUklPUklUWV9OT1JNQUw7DQo+IC0JZWxzZQ0KPiAtCQlwcmlvcml0 eSA9IFJQQ19QUklPUklUWV9ISUdIOw0KPiAtCXJwY19zbGVlcF9vbl9wcmlvcml0eSgmeHBydC0+ c2VuZGluZywgdGFzaywgTlVMTCwgcHJpb3JpdHkpOw0KPiArCXJwY19zbGVlcF9vbigmeHBydC0+ c2VuZGluZywgdGFzaywgTlVMTCk7DQo+ICAJcmV0dXJuIDA7DQo+ICB9DQo+ICBFWFBPUlRfU1lN Qk9MX0dQTCh4cHJ0X3Jlc2VydmVfeHBydF9jb25nKTsNCj4gQEAgLTI4NSw4ICsyNzEsNyBAQCBz dGF0aWMgdm9pZCBfX3hwcnRfbG9ja193cml0ZV9uZXh0KHN0cnVjdCBycGNfeHBydCAqeHBydCkN Cj4gIAlpZiAodGVzdF9hbmRfc2V0X2JpdChYUFJUX0xPQ0tFRCwgJnhwcnQtPnN0YXRlKSkNCj4g IAkJcmV0dXJuOw0KPiAgDQo+IC0JaWYgKHJwY193YWtlX3VwX2ZpcnN0X29uX3dxKHhwcnRpb2Rf d29ya3F1ZXVlLCAmeHBydC0+c2VuZGluZywNCj4gLQkJCQlfX3hwcnRfbG9ja193cml0ZV9mdW5j LCB4cHJ0KSkNCj4gKwlpZiAocnBjX3dha2VfdXBfZmlyc3QoJnhwcnQtPnNlbmRpbmcsIF9feHBy dF9sb2NrX3dyaXRlX2Z1bmMsIHhwcnQpKQ0KPiAgCQlyZXR1cm47DQo+ICAJeHBydF9jbGVhcl9s b2NrZWQoeHBydCk7DQo+ICB9DQo+IEBAIC0yOTcsOCArMjgyLDcgQEAgc3RhdGljIHZvaWQgX194 cHJ0X2xvY2tfd3JpdGVfbmV4dF9jb25nKHN0cnVjdCBycGNfeHBydA0KPiAqeHBydCkNCj4gIAkJ cmV0dXJuOw0KPiAgCWlmIChSUENYUFJUX0NPTkdFU1RFRCh4cHJ0KSkNCj4gIAkJZ290byBvdXRf dW5sb2NrOw0KPiAtCWlmIChycGNfd2FrZV91cF9maXJzdF9vbl93cSh4cHJ0aW9kX3dvcmtxdWV1 ZSwgJnhwcnQtPnNlbmRpbmcsDQo+IC0JCQkJX194cHJ0X2xvY2tfd3JpdGVfZnVuYywgeHBydCkp DQo+ICsJaWYgKHJwY193YWtlX3VwX2ZpcnN0KCZ4cHJ0LT5zZW5kaW5nLCBfX3hwcnRfbG9ja193 cml0ZV9mdW5jLCB4cHJ0KSkNCj4gIAkJcmV0dXJuOw0KPiAgb3V0X3VubG9jazoNCj4gIAl4cHJ0 X2NsZWFyX2xvY2tlZCh4cHJ0KTsNCj4gQEAgLTQ5NSw4ICs0NzksNyBAQCB2b2lkIHhwcnRfd3Jp dGVfc3BhY2Uoc3RydWN0IHJwY194cHJ0ICp4cHJ0KQ0KPiAgCWlmICh4cHJ0LT5zbmRfdGFzaykg ew0KPiAgCQlkcHJpbnRrKCJSUEM6ICAgICAgIHdyaXRlIHNwYWNlOiB3YWtpbmcgd2FpdGluZyB0 YXNrIG9uICINCj4gIAkJCQkieHBydCAlcFxuIiwgeHBydCk7DQo+IC0JCXJwY193YWtlX3VwX3F1 ZXVlZF90YXNrX29uX3dxKHhwcnRpb2Rfd29ya3F1ZXVlLA0KPiAtCQkJCSZ4cHJ0LT5wZW5kaW5n LCB4cHJ0LT5zbmRfdGFzayk7DQo+ICsJCXJwY193YWtlX3VwX3F1ZXVlZF90YXNrKCZ4cHJ0LT5w ZW5kaW5nLCB4cHJ0LT5zbmRfdGFzayk7DQo+ICAJfQ0KPiAgCXNwaW5fdW5sb2NrX2JoKCZ4cHJ0 LT50cmFuc3BvcnRfbG9jayk7DQo+ICB9DQo+IEBAIC0xNTk2LDcgKzE1NzksNyBAQCBzdGF0aWMg dm9pZCB4cHJ0X2luaXQoc3RydWN0IHJwY194cHJ0ICp4cHJ0LCBzdHJ1Y3QgbmV0DQo+ICpuZXQp DQo+ICANCj4gIAlycGNfaW5pdF93YWl0X3F1ZXVlKCZ4cHJ0LT5iaW5kaW5nLCAieHBydF9iaW5k aW5nIik7DQo+ICAJcnBjX2luaXRfd2FpdF9xdWV1ZSgmeHBydC0+cGVuZGluZywgInhwcnRfcGVu ZGluZyIpOw0KPiAtCXJwY19pbml0X3ByaW9yaXR5X3dhaXRfcXVldWUoJnhwcnQtPnNlbmRpbmcs ICJ4cHJ0X3NlbmRpbmciKTsNCj4gKwlycGNfaW5pdF93YWl0X3F1ZXVlKCZ4cHJ0LT5zZW5kaW5n LCAieHBydF9zZW5kaW5nIik7DQo+ICAJcnBjX2luaXRfcHJpb3JpdHlfd2FpdF9xdWV1ZSgmeHBy dC0+YmFja2xvZywgInhwcnRfYmFja2xvZyIpOw0KPiAgDQo+ICAJeHBydF9pbml0X3hpZCh4cHJ0 KTsNCg== ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue 2018-09-06 14:17 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Schumaker, Anna @ 2018-09-06 14:23 ` Schumaker, Anna 2018-09-06 14:23 ` Schumaker, Anna 1 sibling, 0 replies; 45+ messages in thread From: Schumaker, Anna @ 2018-09-06 14:23 UTC (permalink / raw) To: linux-nfs, trondmy T24gVGh1LCAyMDE4LTA5LTA2IGF0IDEwOjE3IC0wNDAwLCBBbm5hIFNjaHVtYWtlciB3cm90ZToN Cj4gSGkgVHJvbmQsDQo+IA0KPiBPbiBUdWUsIDIwMTgtMDktMDQgYXQgMTc6MDUgLTA0MDAsIFRy b25kIE15a2xlYnVzdCB3cm90ZToNCj4gPiBXZSBubyBsb25nZXIgbmVlZCBwcmlvcml0eSBzZW1h bnRpY3Mgb24gdGhlIHhwcnQtPnNlbmRpbmcgcXVldWUsIGJlY2F1c2UNCj4gPiB0aGUgb3JkZXIg aW4gd2hpY2ggdGFza3MgYXJlIHNlbnQgaXMgbm93IGRpY3RhdGVkIGJ5IHRoZWlyIHBvc2l0aW9u IGluDQo+ID4gdGhlIHNlbmQgcXVldWUuDQo+ID4gTm90ZSB0aGF0IHRoZSBiYWNrbG9nIHF1ZXVl IHJlbWFpbnMgYSBwcmlvcml0eSBxdWV1ZSwgbWVhbmluZyB0aGF0DQo+ID4gc2xvdCByZXNvdXJj ZXMgYXJlIHN0aWxsIG1hbmFnZWQgaW4gb3JkZXIgb2YgdGFzayBwcmlvcml0eS4NCj4gPiANCj4g PiBTaWduZWQtb2ZmLWJ5OiBUcm9uZCBNeWtsZWJ1c3QgPHRyb25kLm15a2xlYnVzdEBoYW1tZXJz cGFjZS5jb20+DQo+ID4gLS0tDQo+ID4gIG5ldC9zdW5ycGMveHBydC5jIHwgMjkgKysrKysrLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gPiAgMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygr KSwgMjMgZGVsZXRpb25zKC0pDQo+ID4gDQo+ID4gZGlmZiAtLWdpdCBhL25ldC9zdW5ycGMveHBy dC5jIGIvbmV0L3N1bnJwYy94cHJ0LmMNCj4gPiBpbmRleCBiYTlhZjI1ZDE0ZGUuLmViZTUyMzVj NDg0YiAxMDA2NDQNCj4gPiAtLS0gYS9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ICsrKyBiL25ldC9z dW5ycGMveHBydC5jDQo+ID4gQEAgLTE4MSw3ICsxODEsNiBAQCBFWFBPUlRfU1lNQk9MX0dQTCh4 cHJ0X2xvYWRfdHJhbnNwb3J0KTsNCj4gPiAgaW50IHhwcnRfcmVzZXJ2ZV94cHJ0KHN0cnVjdCBy cGNfeHBydCAqeHBydCwgc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+ICB7DQo+ID4gIAlzdHJ1 Y3QgcnBjX3Jxc3QgKnJlcSA9IHRhc2stPnRrX3Jxc3RwOw0KPiANCj4gQXMgb2YgdGhpcyBwYXRj aCwgcmVxIGlzIG5vIGxvbmdlciB1c2VkIGVpdGhlci4gIENhbiBpdCBiZSByZW1vdmVkPw0KDQpO ZXZlciBtaW5kLCBJIGxvb2tlZCBhdCB0aGUgbmV4dCBwYXRjaCBhbmQgc2F3IHRoYXQgaXQncyB1 c2VkIGFnYWluLg0KDQo+IA0KPiBUaGFua3MsDQo+IEFubmENCj4gDQo+ID4gLQlpbnQgcHJpb3Jp dHk7DQo+ID4gIA0KPiA+ICAJaWYgKHRlc3RfYW5kX3NldF9iaXQoWFBSVF9MT0NLRUQsICZ4cHJ0 LT5zdGF0ZSkpIHsNCj4gPiAgCQlpZiAodGFzayA9PSB4cHJ0LT5zbmRfdGFzaykNCj4gPiBAQCAt MTk3LDEzICsxOTYsNyBAQCBpbnQgeHBydF9yZXNlcnZlX3hwcnQoc3RydWN0IHJwY194cHJ0ICp4 cHJ0LCBzdHJ1Y3QNCj4gPiBycGNfdGFzayAqdGFzaykNCj4gPiAgCQkJdGFzay0+dGtfcGlkLCB4 cHJ0KTsNCj4gPiAgCXRhc2stPnRrX3RpbWVvdXQgPSAwOw0KPiA+ICAJdGFzay0+dGtfc3RhdHVz ID0gLUVBR0FJTjsNCj4gPiAtCWlmIChyZXEgPT0gTlVMTCkNCj4gPiAtCQlwcmlvcml0eSA9IFJQ Q19QUklPUklUWV9MT1c7DQo+ID4gLQllbHNlIGlmICghcmVxLT5ycV9udHJhbnMpDQo+ID4gLQkJ cHJpb3JpdHkgPSBSUENfUFJJT1JJVFlfTk9STUFMOw0KPiA+IC0JZWxzZQ0KPiA+IC0JCXByaW9y aXR5ID0gUlBDX1BSSU9SSVRZX0hJR0g7DQo+ID4gLQlycGNfc2xlZXBfb25fcHJpb3JpdHkoJnhw cnQtPnNlbmRpbmcsIHRhc2ssIE5VTEwsIHByaW9yaXR5KTsNCj4gPiArCXJwY19zbGVlcF9vbigm eHBydC0+c2VuZGluZywgdGFzaywgTlVMTCk7DQo+ID4gIAlyZXR1cm4gMDsNCj4gPiAgfQ0KPiA+ ICBFWFBPUlRfU1lNQk9MX0dQTCh4cHJ0X3Jlc2VydmVfeHBydCk7DQo+ID4gQEAgLTIzMSw3ICsy MjQsNiBAQCBzdGF0aWMgdm9pZCB4cHJ0X2NsZWFyX2xvY2tlZChzdHJ1Y3QgcnBjX3hwcnQgKnhw cnQpDQo+ID4gIGludCB4cHJ0X3Jlc2VydmVfeHBydF9jb25nKHN0cnVjdCBycGNfeHBydCAqeHBy dCwgc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+ICB7DQo+ID4gIAlzdHJ1Y3QgcnBjX3Jxc3Qg KnJlcSA9IHRhc2stPnRrX3Jxc3RwOw0KPiA+IC0JaW50IHByaW9yaXR5Ow0KPiA+ICANCj4gPiAg CWlmICh0ZXN0X2FuZF9zZXRfYml0KFhQUlRfTE9DS0VELCAmeHBydC0+c3RhdGUpKSB7DQo+ID4g IAkJaWYgKHRhc2sgPT0geHBydC0+c25kX3Rhc2spDQo+ID4gQEAgLTI1MSwxMyArMjQzLDcgQEAg aW50IHhwcnRfcmVzZXJ2ZV94cHJ0X2Nvbmcoc3RydWN0IHJwY194cHJ0ICp4cHJ0LA0KPiA+IHN0 cnVjdA0KPiA+IHJwY190YXNrICp0YXNrKQ0KPiA+ICAJZHByaW50aygiUlBDOiAlNXUgZmFpbGVk IHRvIGxvY2sgdHJhbnNwb3J0ICVwXG4iLCB0YXNrLT50a19waWQsIHhwcnQpOw0KPiA+ICAJdGFz ay0+dGtfdGltZW91dCA9IDA7DQo+ID4gIAl0YXNrLT50a19zdGF0dXMgPSAtRUFHQUlOOw0KPiA+ IC0JaWYgKHJlcSA9PSBOVUxMKQ0KPiA+IC0JCXByaW9yaXR5ID0gUlBDX1BSSU9SSVRZX0xPVzsN Cj4gPiAtCWVsc2UgaWYgKCFyZXEtPnJxX250cmFucykNCj4gPiAtCQlwcmlvcml0eSA9IFJQQ19Q UklPUklUWV9OT1JNQUw7DQo+ID4gLQllbHNlDQo+ID4gLQkJcHJpb3JpdHkgPSBSUENfUFJJT1JJ VFlfSElHSDsNCj4gPiAtCXJwY19zbGVlcF9vbl9wcmlvcml0eSgmeHBydC0+c2VuZGluZywgdGFz aywgTlVMTCwgcHJpb3JpdHkpOw0KPiA+ICsJcnBjX3NsZWVwX29uKCZ4cHJ0LT5zZW5kaW5nLCB0 YXNrLCBOVUxMKTsNCj4gPiAgCXJldHVybiAwOw0KPiA+ICB9DQo+ID4gIEVYUE9SVF9TWU1CT0xf R1BMKHhwcnRfcmVzZXJ2ZV94cHJ0X2NvbmcpOw0KPiA+IEBAIC0yODUsOCArMjcxLDcgQEAgc3Rh dGljIHZvaWQgX194cHJ0X2xvY2tfd3JpdGVfbmV4dChzdHJ1Y3QgcnBjX3hwcnQNCj4gPiAqeHBy dCkNCj4gPiAgCWlmICh0ZXN0X2FuZF9zZXRfYml0KFhQUlRfTE9DS0VELCAmeHBydC0+c3RhdGUp KQ0KPiA+ICAJCXJldHVybjsNCj4gPiAgDQo+ID4gLQlpZiAocnBjX3dha2VfdXBfZmlyc3Rfb25f d3EoeHBydGlvZF93b3JrcXVldWUsICZ4cHJ0LT5zZW5kaW5nLA0KPiA+IC0JCQkJX194cHJ0X2xv Y2tfd3JpdGVfZnVuYywgeHBydCkpDQo+ID4gKwlpZiAocnBjX3dha2VfdXBfZmlyc3QoJnhwcnQt PnNlbmRpbmcsIF9feHBydF9sb2NrX3dyaXRlX2Z1bmMsIHhwcnQpKQ0KPiA+ICAJCXJldHVybjsN Cj4gPiAgCXhwcnRfY2xlYXJfbG9ja2VkKHhwcnQpOw0KPiA+ICB9DQo+ID4gQEAgLTI5Nyw4ICsy ODIsNyBAQCBzdGF0aWMgdm9pZCBfX3hwcnRfbG9ja193cml0ZV9uZXh0X2Nvbmcoc3RydWN0IHJw Y194cHJ0DQo+ID4gKnhwcnQpDQo+ID4gIAkJcmV0dXJuOw0KPiA+ICAJaWYgKFJQQ1hQUlRfQ09O R0VTVEVEKHhwcnQpKQ0KPiA+ICAJCWdvdG8gb3V0X3VubG9jazsNCj4gPiAtCWlmIChycGNfd2Fr ZV91cF9maXJzdF9vbl93cSh4cHJ0aW9kX3dvcmtxdWV1ZSwgJnhwcnQtPnNlbmRpbmcsDQo+ID4g LQkJCQlfX3hwcnRfbG9ja193cml0ZV9mdW5jLCB4cHJ0KSkNCj4gPiArCWlmIChycGNfd2FrZV91 cF9maXJzdCgmeHBydC0+c2VuZGluZywgX194cHJ0X2xvY2tfd3JpdGVfZnVuYywgeHBydCkpDQo+ ID4gIAkJcmV0dXJuOw0KPiA+ICBvdXRfdW5sb2NrOg0KPiA+ICAJeHBydF9jbGVhcl9sb2NrZWQo eHBydCk7DQo+ID4gQEAgLTQ5NSw4ICs0NzksNyBAQCB2b2lkIHhwcnRfd3JpdGVfc3BhY2Uoc3Ry dWN0IHJwY194cHJ0ICp4cHJ0KQ0KPiA+ICAJaWYgKHhwcnQtPnNuZF90YXNrKSB7DQo+ID4gIAkJ ZHByaW50aygiUlBDOiAgICAgICB3cml0ZSBzcGFjZTogd2FraW5nIHdhaXRpbmcgdGFzayBvbiAi DQo+ID4gIAkJCQkieHBydCAlcFxuIiwgeHBydCk7DQo+ID4gLQkJcnBjX3dha2VfdXBfcXVldWVk X3Rhc2tfb25fd3EoeHBydGlvZF93b3JrcXVldWUsDQo+ID4gLQkJCQkmeHBydC0+cGVuZGluZywg eHBydC0+c25kX3Rhc2spOw0KPiA+ICsJCXJwY193YWtlX3VwX3F1ZXVlZF90YXNrKCZ4cHJ0LT5w ZW5kaW5nLCB4cHJ0LT5zbmRfdGFzayk7DQo+ID4gIAl9DQo+ID4gIAlzcGluX3VubG9ja19iaCgm eHBydC0+dHJhbnNwb3J0X2xvY2spOw0KPiA+ICB9DQo+ID4gQEAgLTE1OTYsNyArMTU3OSw3IEBA IHN0YXRpYyB2b2lkIHhwcnRfaW5pdChzdHJ1Y3QgcnBjX3hwcnQgKnhwcnQsIHN0cnVjdA0KPiA+ IG5ldA0KPiA+ICpuZXQpDQo+ID4gIA0KPiA+ICAJcnBjX2luaXRfd2FpdF9xdWV1ZSgmeHBydC0+ YmluZGluZywgInhwcnRfYmluZGluZyIpOw0KPiA+ICAJcnBjX2luaXRfd2FpdF9xdWV1ZSgmeHBy dC0+cGVuZGluZywgInhwcnRfcGVuZGluZyIpOw0KPiA+IC0JcnBjX2luaXRfcHJpb3JpdHlfd2Fp dF9xdWV1ZSgmeHBydC0+c2VuZGluZywgInhwcnRfc2VuZGluZyIpOw0KPiA+ICsJcnBjX2luaXRf d2FpdF9xdWV1ZSgmeHBydC0+c2VuZGluZywgInhwcnRfc2VuZGluZyIpOw0KPiA+ICAJcnBjX2lu aXRfcHJpb3JpdHlfd2FpdF9xdWV1ZSgmeHBydC0+YmFja2xvZywgInhwcnRfYmFja2xvZyIpOw0K PiA+ICANCj4gPiAgCXhwcnRfaW5pdF94aWQoeHBydCk7DQo= ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue 2018-09-06 14:17 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Schumaker, Anna 2018-09-06 14:23 ` Schumaker, Anna @ 2018-09-06 14:23 ` Schumaker, Anna 1 sibling, 0 replies; 45+ messages in thread From: Schumaker, Anna @ 2018-09-06 14:23 UTC (permalink / raw) To: linux-nfs, trondmy T24gVGh1LCAyMDE4LTA5LTA2IGF0IDEwOjE3IC0wNDAwLCBBbm5hIFNjaHVtYWtlciB3cm90ZToN Cj4gSGkgVHJvbmQsDQo+IA0KPiBPbiBUdWUsIDIwMTgtMDktMDQgYXQgMTc6MDUgLTA0MDAsIFRy b25kIE15a2xlYnVzdCB3cm90ZToNCj4gPiBXZSBubyBsb25nZXIgbmVlZCBwcmlvcml0eSBzZW1h bnRpY3Mgb24gdGhlIHhwcnQtPnNlbmRpbmcgcXVldWUsIGJlY2F1c2UNCj4gPiB0aGUgb3JkZXIg aW4gd2hpY2ggdGFza3MgYXJlIHNlbnQgaXMgbm93IGRpY3RhdGVkIGJ5IHRoZWlyIHBvc2l0aW9u IGluDQo+ID4gdGhlIHNlbmQgcXVldWUuDQo+ID4gTm90ZSB0aGF0IHRoZSBiYWNrbG9nIHF1ZXVl IHJlbWFpbnMgYSBwcmlvcml0eSBxdWV1ZSwgbWVhbmluZyB0aGF0DQo+ID4gc2xvdCByZXNvdXJj ZXMgYXJlIHN0aWxsIG1hbmFnZWQgaW4gb3JkZXIgb2YgdGFzayBwcmlvcml0eS4NCj4gPiANCj4g PiBTaWduZWQtb2ZmLWJ5OiBUcm9uZCBNeWtsZWJ1c3QgPHRyb25kLm15a2xlYnVzdEBoYW1tZXJz cGFjZS5jb20+DQo+ID4gLS0tDQo+ID4gIG5ldC9zdW5ycGMveHBydC5jIHwgMjkgKysrKysrLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gPiAgMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygr KSwgMjMgZGVsZXRpb25zKC0pDQo+ID4gDQo+ID4gZGlmZiAtLWdpdCBhL25ldC9zdW5ycGMveHBy dC5jIGIvbmV0L3N1bnJwYy94cHJ0LmMNCj4gPiBpbmRleCBiYTlhZjI1ZDE0ZGUuLmViZTUyMzVj NDg0YiAxMDA2NDQNCj4gPiAtLS0gYS9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ICsrKyBiL25ldC9z dW5ycGMveHBydC5jDQo+ID4gQEAgLTE4MSw3ICsxODEsNiBAQCBFWFBPUlRfU1lNQk9MX0dQTCh4 cHJ0X2xvYWRfdHJhbnNwb3J0KTsNCj4gPiAgaW50IHhwcnRfcmVzZXJ2ZV94cHJ0KHN0cnVjdCBy cGNfeHBydCAqeHBydCwgc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+ICB7DQo+ID4gIAlzdHJ1 Y3QgcnBjX3Jxc3QgKnJlcSA9IHRhc2stPnRrX3Jxc3RwOw0KPiANCj4gQXMgb2YgdGhpcyBwYXRj aCwgcmVxIGlzIG5vIGxvbmdlciB1c2VkIGVpdGhlci4gIENhbiBpdCBiZSByZW1vdmVkPw0KDQpO ZXZlciBtaW5kLCBJIGxvb2tlZCBhdCB0aGUgbmV4dCBwYXRjaCBhbmQgc2F3IHRoYXQgaXQncyB1 c2VkIGFnYWluLg0KDQo+IA0KPiBUaGFua3MsDQo+IEFubmENCj4gDQo+ID4gLQlpbnQgcHJpb3Jp dHk7DQo+ID4gIA0KPiA+ICAJaWYgKHRlc3RfYW5kX3NldF9iaXQoWFBSVF9MT0NLRUQsICZ4cHJ0 LT5zdGF0ZSkpIHsNCj4gPiAgCQlpZiAodGFzayA9PSB4cHJ0LT5zbmRfdGFzaykNCj4gPiBAQCAt MTk3LDEzICsxOTYsNyBAQCBpbnQgeHBydF9yZXNlcnZlX3hwcnQoc3RydWN0IHJwY194cHJ0ICp4 cHJ0LCBzdHJ1Y3QNCj4gPiBycGNfdGFzayAqdGFzaykNCj4gPiAgCQkJdGFzay0+dGtfcGlkLCB4 cHJ0KTsNCj4gPiAgCXRhc2stPnRrX3RpbWVvdXQgPSAwOw0KPiA+ICAJdGFzay0+dGtfc3RhdHVz ID0gLUVBR0FJTjsNCj4gPiAtCWlmIChyZXEgPT0gTlVMTCkNCj4gPiAtCQlwcmlvcml0eSA9IFJQ Q19QUklPUklUWV9MT1c7DQo+ID4gLQllbHNlIGlmICghcmVxLT5ycV9udHJhbnMpDQo+ID4gLQkJ cHJpb3JpdHkgPSBSUENfUFJJT1JJVFlfTk9STUFMOw0KPiA+IC0JZWxzZQ0KPiA+IC0JCXByaW9y aXR5ID0gUlBDX1BSSU9SSVRZX0hJR0g7DQo+ID4gLQlycGNfc2xlZXBfb25fcHJpb3JpdHkoJnhw cnQtPnNlbmRpbmcsIHRhc2ssIE5VTEwsIHByaW9yaXR5KTsNCj4gPiArCXJwY19zbGVlcF9vbigm eHBydC0+c2VuZGluZywgdGFzaywgTlVMTCk7DQo+ID4gIAlyZXR1cm4gMDsNCj4gPiAgfQ0KPiA+ ICBFWFBPUlRfU1lNQk9MX0dQTCh4cHJ0X3Jlc2VydmVfeHBydCk7DQo+ID4gQEAgLTIzMSw3ICsy MjQsNiBAQCBzdGF0aWMgdm9pZCB4cHJ0X2NsZWFyX2xvY2tlZChzdHJ1Y3QgcnBjX3hwcnQgKnhw cnQpDQo+ID4gIGludCB4cHJ0X3Jlc2VydmVfeHBydF9jb25nKHN0cnVjdCBycGNfeHBydCAqeHBy dCwgc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+ICB7DQo+ID4gIAlzdHJ1Y3QgcnBjX3Jxc3Qg KnJlcSA9IHRhc2stPnRrX3Jxc3RwOw0KPiA+IC0JaW50IHByaW9yaXR5Ow0KPiA+ICANCj4gPiAg CWlmICh0ZXN0X2FuZF9zZXRfYml0KFhQUlRfTE9DS0VELCAmeHBydC0+c3RhdGUpKSB7DQo+ID4g IAkJaWYgKHRhc2sgPT0geHBydC0+c25kX3Rhc2spDQo+ID4gQEAgLTI1MSwxMyArMjQzLDcgQEAg aW50IHhwcnRfcmVzZXJ2ZV94cHJ0X2Nvbmcoc3RydWN0IHJwY194cHJ0ICp4cHJ0LA0KPiA+IHN0 cnVjdA0KPiA+IHJwY190YXNrICp0YXNrKQ0KPiA+ICAJZHByaW50aygiUlBDOiAlNXUgZmFpbGVk IHRvIGxvY2sgdHJhbnNwb3J0ICVwXG4iLCB0YXNrLT50a19waWQsIHhwcnQpOw0KPiA+ICAJdGFz ay0+dGtfdGltZW91dCA9IDA7DQo+ID4gIAl0YXNrLT50a19zdGF0dXMgPSAtRUFHQUlOOw0KPiA+ IC0JaWYgKHJlcSA9PSBOVUxMKQ0KPiA+IC0JCXByaW9yaXR5ID0gUlBDX1BSSU9SSVRZX0xPVzsN Cj4gPiAtCWVsc2UgaWYgKCFyZXEtPnJxX250cmFucykNCj4gPiAtCQlwcmlvcml0eSA9IFJQQ19Q UklPUklUWV9OT1JNQUw7DQo+ID4gLQllbHNlDQo+ID4gLQkJcHJpb3JpdHkgPSBSUENfUFJJT1JJ VFlfSElHSDsNCj4gPiAtCXJwY19zbGVlcF9vbl9wcmlvcml0eSgmeHBydC0+c2VuZGluZywgdGFz aywgTlVMTCwgcHJpb3JpdHkpOw0KPiA+ICsJcnBjX3NsZWVwX29uKCZ4cHJ0LT5zZW5kaW5nLCB0 YXNrLCBOVUxMKTsNCj4gPiAgCXJldHVybiAwOw0KPiA+ICB9DQo+ID4gIEVYUE9SVF9TWU1CT0xf R1BMKHhwcnRfcmVzZXJ2ZV94cHJ0X2NvbmcpOw0KPiA+IEBAIC0yODUsOCArMjcxLDcgQEAgc3Rh dGljIHZvaWQgX194cHJ0X2xvY2tfd3JpdGVfbmV4dChzdHJ1Y3QgcnBjX3hwcnQNCj4gPiAqeHBy dCkNCj4gPiAgCWlmICh0ZXN0X2FuZF9zZXRfYml0KFhQUlRfTE9DS0VELCAmeHBydC0+c3RhdGUp KQ0KPiA+ICAJCXJldHVybjsNCj4gPiAgDQo+ID4gLQlpZiAocnBjX3dha2VfdXBfZmlyc3Rfb25f d3EoeHBydGlvZF93b3JrcXVldWUsICZ4cHJ0LT5zZW5kaW5nLA0KPiA+IC0JCQkJX194cHJ0X2xv Y2tfd3JpdGVfZnVuYywgeHBydCkpDQo+ID4gKwlpZiAocnBjX3dha2VfdXBfZmlyc3QoJnhwcnQt PnNlbmRpbmcsIF9feHBydF9sb2NrX3dyaXRlX2Z1bmMsIHhwcnQpKQ0KPiA+ICAJCXJldHVybjsN Cj4gPiAgCXhwcnRfY2xlYXJfbG9ja2VkKHhwcnQpOw0KPiA+ICB9DQo+ID4gQEAgLTI5Nyw4ICsy ODIsNyBAQCBzdGF0aWMgdm9pZCBfX3hwcnRfbG9ja193cml0ZV9uZXh0X2Nvbmcoc3RydWN0IHJw Y194cHJ0DQo+ID4gKnhwcnQpDQo+ID4gIAkJcmV0dXJuOw0KPiA+ICAJaWYgKFJQQ1hQUlRfQ09O R0VTVEVEKHhwcnQpKQ0KPiA+ICAJCWdvdG8gb3V0X3VubG9jazsNCj4gPiAtCWlmIChycGNfd2Fr ZV91cF9maXJzdF9vbl93cSh4cHJ0aW9kX3dvcmtxdWV1ZSwgJnhwcnQtPnNlbmRpbmcsDQo+ID4g LQkJCQlfX3hwcnRfbG9ja193cml0ZV9mdW5jLCB4cHJ0KSkNCj4gPiArCWlmIChycGNfd2FrZV91 cF9maXJzdCgmeHBydC0+c2VuZGluZywgX194cHJ0X2xvY2tfd3JpdGVfZnVuYywgeHBydCkpDQo+ ID4gIAkJcmV0dXJuOw0KPiA+ICBvdXRfdW5sb2NrOg0KPiA+ICAJeHBydF9jbGVhcl9sb2NrZWQo eHBydCk7DQo+ID4gQEAgLTQ5NSw4ICs0NzksNyBAQCB2b2lkIHhwcnRfd3JpdGVfc3BhY2Uoc3Ry dWN0IHJwY194cHJ0ICp4cHJ0KQ0KPiA+ICAJaWYgKHhwcnQtPnNuZF90YXNrKSB7DQo+ID4gIAkJ ZHByaW50aygiUlBDOiAgICAgICB3cml0ZSBzcGFjZTogd2FraW5nIHdhaXRpbmcgdGFzayBvbiAi DQo+ID4gIAkJCQkieHBydCAlcFxuIiwgeHBydCk7DQo+ID4gLQkJcnBjX3dha2VfdXBfcXVldWVk X3Rhc2tfb25fd3EoeHBydGlvZF93b3JrcXVldWUsDQo+ID4gLQkJCQkmeHBydC0+cGVuZGluZywg eHBydC0+c25kX3Rhc2spOw0KPiA+ICsJCXJwY193YWtlX3VwX3F1ZXVlZF90YXNrKCZ4cHJ0LT5w ZW5kaW5nLCB4cHJ0LT5zbmRfdGFzayk7DQo+ID4gIAl9DQo+ID4gIAlzcGluX3VubG9ja19iaCgm eHBydC0+dHJhbnNwb3J0X2xvY2spOw0KPiA+ICB9DQo+ID4gQEAgLTE1OTYsNyArMTU3OSw3IEBA IHN0YXRpYyB2b2lkIHhwcnRfaW5pdChzdHJ1Y3QgcnBjX3hwcnQgKnhwcnQsIHN0cnVjdA0KPiA+ IG5ldA0KPiA+ICpuZXQpDQo+ID4gIA0KPiA+ICAJcnBjX2luaXRfd2FpdF9xdWV1ZSgmeHBydC0+ YmluZGluZywgInhwcnRfYmluZGluZyIpOw0KPiA+ICAJcnBjX2luaXRfd2FpdF9xdWV1ZSgmeHBy dC0+cGVuZGluZywgInhwcnRfcGVuZGluZyIpOw0KPiA+IC0JcnBjX2luaXRfcHJpb3JpdHlfd2Fp dF9xdWV1ZSgmeHBydC0+c2VuZGluZywgInhwcnRfc2VuZGluZyIpOw0KPiA+ICsJcnBjX2luaXRf d2FpdF9xdWV1ZSgmeHBydC0+c2VuZGluZywgInhwcnRfc2VuZGluZyIpOw0KPiA+ICAJcnBjX2lu aXRfcHJpb3JpdHlfd2FpdF9xdWV1ZSgmeHBydC0+YmFja2xvZywgInhwcnRfYmFja2xvZyIpOw0K PiA+ICANCj4gPiAgCXhwcnRfaW5pdF94aWQoeHBydCk7DQo= ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() 2018-09-04 21:05 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 24/34] SUNRPC: Fix up the back channel transmit Trond Myklebust @ 2018-09-05 14:30 ` Chuck Lever 2018-09-05 15:28 ` Trond Myklebust 1 sibling, 1 reply; 45+ messages in thread From: Chuck Lever @ 2018-09-05 14:30 UTC (permalink / raw) To: Trond Myklebust; +Cc: Linux NFS Mailing List > On Sep 4, 2018, at 5:05 PM, Trond Myklebust <trondmy@gmail.com> wrote: >=20 > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> > --- > net/sunrpc/clnt.c | 6 ------ > net/sunrpc/xprt.c | 13 ++++++------- > 2 files changed, 6 insertions(+), 13 deletions(-) >=20 > diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c > index 032b7042adb6..21713bed812a 100644 > --- a/net/sunrpc/clnt.c > +++ b/net/sunrpc/clnt.c > @@ -1967,8 +1967,6 @@ call_connect_status(struct rpc_task *task) > static void > call_transmit(struct rpc_task *task) > { > - int is_retrans =3D RPC_WAS_SENT(task); > - > dprint_status(task); >=20 > task->tk_action =3D call_transmit_status; > @@ -1979,10 +1977,6 @@ call_transmit(struct rpc_task *task) > if (!xprt_prepare_transmit(task)) > return; > xprt_transmit(task); > - if (task->tk_status < 0) > - return; > - if (is_retrans) > - task->tk_client->cl_stats->rpcretrans++; > } >=20 > /* > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c > index f1301d391399..e2f5b4668a66 100644 > --- a/net/sunrpc/xprt.c > +++ b/net/sunrpc/xprt.c > @@ -191,8 +191,6 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, = struct rpc_task *task) > goto out_sleep; > } > xprt->snd_task =3D task; > - if (req !=3D NULL) > - req->rq_ntrans++; >=20 > return 1; >=20 > @@ -247,7 +245,6 @@ int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, = struct rpc_task *task) > } > if (__xprt_get_cong(xprt, task)) { > xprt->snd_task =3D task; > - req->rq_ntrans++; > return 1; > } > xprt_clear_locked(xprt); > @@ -281,12 +278,8 @@ static inline int xprt_lock_write(struct rpc_xprt = *xprt, struct rpc_task *task) > static bool __xprt_lock_write_func(struct rpc_task *task, void *data) > { > struct rpc_xprt *xprt =3D data; > - struct rpc_rqst *req; >=20 > - req =3D task->tk_rqstp; > xprt->snd_task =3D task; > - if (req) > - req->rq_ntrans++; > return true; > } >=20 > @@ -1126,6 +1119,7 @@ void xprt_transmit(struct rpc_task *task) > struct rpc_rqst *req =3D task->tk_rqstp; > struct rpc_xprt *xprt =3D req->rq_xprt; > unsigned int connect_cookie; > + int is_retrans =3D RPC_WAS_SENT(task); > int status; >=20 > dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, = req->rq_slen); > @@ -1140,6 +1134,8 @@ void xprt_transmit(struct rpc_task *task) > } > } >=20 > + req->rq_ntrans++; > + rq_ntrans is used in two places: 1. rpc_update_rtt 2. rpc_count_iostats_metrics Both of these appear to assume that rq_ntrans is counting full successful transmissions (ie, calls that appear on the wire), not invocations of ->send_request(). Can this counter be moved down to where rpcretrans is updated? > connect_cookie =3D xprt->connect_cookie; > status =3D xprt->ops->send_request(req, task); > trace_xprt_transmit(xprt, req->rq_xid, status); > @@ -1148,6 +1144,9 @@ void xprt_transmit(struct rpc_task *task) > return; > } >=20 > + if (is_retrans) > + task->tk_client->cl_stats->rpcretrans++; > + > xprt_inject_disconnect(xprt); >=20 > dprintk("RPC: %5u xmit complete\n", task->tk_pid); > --=20 > 2.17.1 >=20 -- Chuck Lever ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() 2018-09-05 14:30 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Chuck Lever @ 2018-09-05 15:28 ` Trond Myklebust 2018-09-05 15:31 ` Chuck Lever 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-05 15:28 UTC (permalink / raw) To: chuck.lever; +Cc: linux-nfs T24gV2VkLCAyMDE4LTA5LTA1IGF0IDEwOjMwIC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g PiBPbiBTZXAgNCwgMjAxOCwgYXQgNTowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QGdt YWlsLmNvbT4NCj4gPiB3cm90ZToNCj4gPiANCj4gPiBTaWduZWQtb2ZmLWJ5OiBUcm9uZCBNeWts ZWJ1c3QgPHRyb25kLm15a2xlYnVzdEBoYW1tZXJzcGFjZS5jb20+DQo+ID4gLS0tDQo+ID4gbmV0 L3N1bnJwYy9jbG50LmMgfCAgNiAtLS0tLS0NCj4gPiBuZXQvc3VucnBjL3hwcnQuYyB8IDEzICsr KysrKy0tLS0tLS0NCj4gPiAyIGZpbGVzIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKSwgMTMgZGVs ZXRpb25zKC0pDQo+ID4gDQo+ID4gZGlmZiAtLWdpdCBhL25ldC9zdW5ycGMvY2xudC5jIGIvbmV0 L3N1bnJwYy9jbG50LmMNCj4gPiBpbmRleCAwMzJiNzA0MmFkYjYuLjIxNzEzYmVkODEyYSAxMDA2 NDQNCj4gPiAtLS0gYS9uZXQvc3VucnBjL2NsbnQuYw0KPiA+ICsrKyBiL25ldC9zdW5ycGMvY2xu dC5jDQo+ID4gQEAgLTE5NjcsOCArMTk2Nyw2IEBAIGNhbGxfY29ubmVjdF9zdGF0dXMoc3RydWN0 IHJwY190YXNrICp0YXNrKQ0KPiA+IHN0YXRpYyB2b2lkDQo+ID4gY2FsbF90cmFuc21pdChzdHJ1 Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ID4gew0KPiA+IC0JaW50IGlzX3JldHJhbnMgPSBSUENfV0FT X1NFTlQodGFzayk7DQo+ID4gLQ0KPiA+IAlkcHJpbnRfc3RhdHVzKHRhc2spOw0KPiA+IA0KPiA+ IAl0YXNrLT50a19hY3Rpb24gPSBjYWxsX3RyYW5zbWl0X3N0YXR1czsNCj4gPiBAQCAtMTk3OSwx MCArMTk3Nyw2IEBAIGNhbGxfdHJhbnNtaXQoc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+IAlp ZiAoIXhwcnRfcHJlcGFyZV90cmFuc21pdCh0YXNrKSkNCj4gPiAJCXJldHVybjsNCj4gPiAJeHBy dF90cmFuc21pdCh0YXNrKTsNCj4gPiAtCWlmICh0YXNrLT50a19zdGF0dXMgPCAwKQ0KPiA+IC0J CXJldHVybjsNCj4gPiAtCWlmIChpc19yZXRyYW5zKQ0KPiA+IC0JCXRhc2stPnRrX2NsaWVudC0+ Y2xfc3RhdHMtPnJwY3JldHJhbnMrKzsNCj4gPiB9DQo+ID4gDQo+ID4gLyoNCj4gPiBkaWZmIC0t Z2l0IGEvbmV0L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+IGluZGV4IGYx MzAxZDM5MTM5OS4uZTJmNWI0NjY4YTY2IDEwMDY0NA0KPiA+IC0tLSBhL25ldC9zdW5ycGMveHBy dC5jDQo+ID4gKysrIGIvbmV0L3N1bnJwYy94cHJ0LmMNCj4gPiBAQCAtMTkxLDggKzE5MSw2IEBA IGludCB4cHJ0X3Jlc2VydmVfeHBydChzdHJ1Y3QgcnBjX3hwcnQgKnhwcnQsDQo+ID4gc3RydWN0 IHJwY190YXNrICp0YXNrKQ0KPiA+IAkJZ290byBvdXRfc2xlZXA7DQo+ID4gCX0NCj4gPiAJeHBy dC0+c25kX3Rhc2sgPSB0YXNrOw0KPiA+IC0JaWYgKHJlcSAhPSBOVUxMKQ0KPiA+IC0JCXJlcS0+ cnFfbnRyYW5zKys7DQo+ID4gDQo+ID4gCXJldHVybiAxOw0KPiA+IA0KPiA+IEBAIC0yNDcsNyAr MjQ1LDYgQEAgaW50IHhwcnRfcmVzZXJ2ZV94cHJ0X2Nvbmcoc3RydWN0IHJwY194cHJ0DQo+ID4g KnhwcnQsIHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4gPiAJfQ0KPiA+IAlpZiAoX194cHJ0X2dl dF9jb25nKHhwcnQsIHRhc2spKSB7DQo+ID4gCQl4cHJ0LT5zbmRfdGFzayA9IHRhc2s7DQo+ID4g LQkJcmVxLT5ycV9udHJhbnMrKzsNCj4gPiAJCXJldHVybiAxOw0KPiA+IAl9DQo+ID4gCXhwcnRf Y2xlYXJfbG9ja2VkKHhwcnQpOw0KPiA+IEBAIC0yODEsMTIgKzI3OCw4IEBAIHN0YXRpYyBpbmxp bmUgaW50IHhwcnRfbG9ja193cml0ZShzdHJ1Y3QNCj4gPiBycGNfeHBydCAqeHBydCwgc3RydWN0 IHJwY190YXNrICp0YXNrKQ0KPiA+IHN0YXRpYyBib29sIF9feHBydF9sb2NrX3dyaXRlX2Z1bmMo c3RydWN0IHJwY190YXNrICp0YXNrLCB2b2lkDQo+ID4gKmRhdGEpDQo+ID4gew0KPiA+IAlzdHJ1 Y3QgcnBjX3hwcnQgKnhwcnQgPSBkYXRhOw0KPiA+IC0Jc3RydWN0IHJwY19ycXN0ICpyZXE7DQo+ ID4gDQo+ID4gLQlyZXEgPSB0YXNrLT50a19ycXN0cDsNCj4gPiAJeHBydC0+c25kX3Rhc2sgPSB0 YXNrOw0KPiA+IC0JaWYgKHJlcSkNCj4gPiAtCQlyZXEtPnJxX250cmFucysrOw0KPiA+IAlyZXR1 cm4gdHJ1ZTsNCj4gPiB9DQo+ID4gDQo+ID4gQEAgLTExMjYsNiArMTExOSw3IEBAIHZvaWQgeHBy dF90cmFuc21pdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ID4gCXN0cnVjdCBycGNfcnFzdAkq cmVxID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gCXN0cnVjdCBycGNfeHBydAkqeHBydCA9IHJlcS0+ cnFfeHBydDsNCj4gPiAJdW5zaWduZWQgaW50IGNvbm5lY3RfY29va2llOw0KPiA+ICsJaW50IGlz X3JldHJhbnMgPSBSUENfV0FTX1NFTlQodGFzayk7DQo+ID4gCWludCBzdGF0dXM7DQo+ID4gDQo+ ID4gCWRwcmludGsoIlJQQzogJTV1IHhwcnRfdHJhbnNtaXQoJXUpXG4iLCB0YXNrLT50a19waWQs IHJlcS0NCj4gPiA+cnFfc2xlbik7DQo+ID4gQEAgLTExNDAsNiArMTEzNCw4IEBAIHZvaWQgeHBy dF90cmFuc21pdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ID4gCQl9DQo+ID4gCX0NCj4gPiAN Cj4gPiArCXJlcS0+cnFfbnRyYW5zKys7DQo+ID4gKw0KPiANCj4gcnFfbnRyYW5zIGlzIHVzZWQg aW4gdHdvIHBsYWNlczoNCj4gDQo+IDEuIHJwY191cGRhdGVfcnR0DQo+IDIuIHJwY19jb3VudF9p b3N0YXRzX21ldHJpY3MNCj4gDQo+IEJvdGggb2YgdGhlc2UgYXBwZWFyIHRvIGFzc3VtZSB0aGF0 IHJxX250cmFucyBpcyBjb3VudGluZw0KPiBmdWxsIHN1Y2Nlc3NmdWwgdHJhbnNtaXNzaW9ucyAo aWUsIGNhbGxzIHRoYXQgYXBwZWFyIG9uIHRoZQ0KPiB3aXJlKSwgbm90IGludm9jYXRpb25zIG9m IC0+c2VuZF9yZXF1ZXN0KCkuDQo+IA0KPiBDYW4gdGhpcyBjb3VudGVyIGJlIG1vdmVkIGRvd24g dG8gd2hlcmUgcnBjcmV0cmFucyBpcyB1cGRhdGVkPw0KPiANCg0KVGhlIHJlYXNvbiB3aHkgSSBk b24ndCB3YW50IHRvIHVwZGF0ZSByZXEtPnJxX250cmFucyBhZnRlciB0cmFuc21pc3Npb24NCmlz IGJlY2F1c2UgdGhhdCB3b3VsZCByYWNlIHdpdGggdGhlIHJlcGx5IGZyb20gdGhlIHNlcnZlciAo d2hpY2ggaXMgbm90DQpibG9ja2VkIGJ5IHRoZSBYUFJUX0xPQ0spLiBJbiB0aGF0IHJhY2Ugc2Nl bmFyaW8sIHJwY191cGRhdGVfcnR0KCkNCmNvdWxkIHVwZGF0ZSB0aGUgUlRUIHZhbHVlIGJlZm9y ZSB3ZSd2ZSB1cGRhdGVkIHRoZSByZXEtPnJxX250cmFucywNCm1lYW5pbmcgdGhhdCB3ZSBtaWdo dCBiZSBtZWFzdXJpbmcgdGhlIHJlc3BvbnNlIHRpbWUgYWdhaW5zdCBhIHJlcGx5IHRvDQp0aGUg b3JpZ2luYWwgdHJhbnNtaXNzaW9uIG9yIHRvIHRoZSByZXRyYW5zbWlzc2lvbiB0aGF0IHdhcyBq dXN0IHNlbnQuDQpUaGF0IHdvdWxkIGVuZCB1cCBicmVha2luZyBWYW4gSmFjb2JzZW4gY29uZ2Vz dGlvbiBjb250cm9sLCBzaW5jZSB3ZQ0KcmlzayBzaWduaWZpY2FudGx5IHVuZGVyZXN0aW1hdGlu ZyB0aGUgUlRUIHZhbHVlLg0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVu dCBtYWludGFpbmVyLCBIYW1tZXJzcGFjZQ0KdHJvbmQubXlrbGVidXN0QGhhbW1lcnNwYWNlLmNv bQ0KDQoNCg== ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() 2018-09-05 15:28 ` Trond Myklebust @ 2018-09-05 15:31 ` Chuck Lever 2018-09-05 16:07 ` Trond Myklebust 0 siblings, 1 reply; 45+ messages in thread From: Chuck Lever @ 2018-09-05 15:31 UTC (permalink / raw) To: Trond Myklebust; +Cc: Linux NFS Mailing List > On Sep 5, 2018, at 11:28 AM, Trond Myklebust <trondmy@hammerspace.com> = wrote: >=20 > On Wed, 2018-09-05 at 10:30 -0400, Chuck Lever wrote: >>> On Sep 4, 2018, at 5:05 PM, Trond Myklebust <trondmy@gmail.com> >>> wrote: >>>=20 >>> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> >>> --- >>> net/sunrpc/clnt.c | 6 ------ >>> net/sunrpc/xprt.c | 13 ++++++------- >>> 2 files changed, 6 insertions(+), 13 deletions(-) >>>=20 >>> diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c >>> index 032b7042adb6..21713bed812a 100644 >>> --- a/net/sunrpc/clnt.c >>> +++ b/net/sunrpc/clnt.c >>> @@ -1967,8 +1967,6 @@ call_connect_status(struct rpc_task *task) >>> static void >>> call_transmit(struct rpc_task *task) >>> { >>> - int is_retrans =3D RPC_WAS_SENT(task); >>> - >>> dprint_status(task); >>>=20 >>> task->tk_action =3D call_transmit_status; >>> @@ -1979,10 +1977,6 @@ call_transmit(struct rpc_task *task) >>> if (!xprt_prepare_transmit(task)) >>> return; >>> xprt_transmit(task); >>> - if (task->tk_status < 0) >>> - return; >>> - if (is_retrans) >>> - task->tk_client->cl_stats->rpcretrans++; >>> } >>>=20 >>> /* >>> diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c >>> index f1301d391399..e2f5b4668a66 100644 >>> --- a/net/sunrpc/xprt.c >>> +++ b/net/sunrpc/xprt.c >>> @@ -191,8 +191,6 @@ int xprt_reserve_xprt(struct rpc_xprt *xprt, >>> struct rpc_task *task) >>> goto out_sleep; >>> } >>> xprt->snd_task =3D task; >>> - if (req !=3D NULL) >>> - req->rq_ntrans++; >>>=20 >>> return 1; >>>=20 >>> @@ -247,7 +245,6 @@ int xprt_reserve_xprt_cong(struct rpc_xprt >>> *xprt, struct rpc_task *task) >>> } >>> if (__xprt_get_cong(xprt, task)) { >>> xprt->snd_task =3D task; >>> - req->rq_ntrans++; >>> return 1; >>> } >>> xprt_clear_locked(xprt); >>> @@ -281,12 +278,8 @@ static inline int xprt_lock_write(struct >>> rpc_xprt *xprt, struct rpc_task *task) >>> static bool __xprt_lock_write_func(struct rpc_task *task, void >>> *data) >>> { >>> struct rpc_xprt *xprt =3D data; >>> - struct rpc_rqst *req; >>>=20 >>> - req =3D task->tk_rqstp; >>> xprt->snd_task =3D task; >>> - if (req) >>> - req->rq_ntrans++; >>> return true; >>> } >>>=20 >>> @@ -1126,6 +1119,7 @@ void xprt_transmit(struct rpc_task *task) >>> struct rpc_rqst *req =3D task->tk_rqstp; >>> struct rpc_xprt *xprt =3D req->rq_xprt; >>> unsigned int connect_cookie; >>> + int is_retrans =3D RPC_WAS_SENT(task); >>> int status; >>>=20 >>> dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, req- >>>> rq_slen); >>> @@ -1140,6 +1134,8 @@ void xprt_transmit(struct rpc_task *task) >>> } >>> } >>>=20 >>> + req->rq_ntrans++; >>> + >>=20 >> rq_ntrans is used in two places: >>=20 >> 1. rpc_update_rtt >> 2. rpc_count_iostats_metrics >>=20 >> Both of these appear to assume that rq_ntrans is counting >> full successful transmissions (ie, calls that appear on the >> wire), not invocations of ->send_request(). >>=20 >> Can this counter be moved down to where rpcretrans is updated? >>=20 >=20 > The reason why I don't want to update req->rq_ntrans after = transmission > is because that would race with the reply from the server (which is = not > blocked by the XPRT_LOCK). In that race scenario, rpc_update_rtt() > could update the RTT value before we've updated the req->rq_ntrans, > meaning that we might be measuring the response time against a reply = to > the original transmission or to the retransmission that was just sent. > That would end up breaking Van Jacobsen congestion control, since we > risk significantly underestimating the RTT value. But for transports that don't use xprt_update_rtt, error returns from ->send_request() (like -EAGAIN or -ENOBUFS) means that rq_ntrans is counted twice, and it shows up as false retransmissions. What if rq_ntrans was bumped in the transport code instead? -- Chuck Lever ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() 2018-09-05 15:31 ` Chuck Lever @ 2018-09-05 16:07 ` Trond Myklebust 2018-09-05 16:34 ` Chuck Lever 0 siblings, 1 reply; 45+ messages in thread From: Trond Myklebust @ 2018-09-05 16:07 UTC (permalink / raw) To: chuck.lever; +Cc: linux-nfs T24gV2VkLCAyMDE4LTA5LTA1IGF0IDExOjMxIC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g PiBPbiBTZXAgNSwgMjAxOCwgYXQgMTE6MjggQU0sIFRyb25kIE15a2xlYnVzdCA8DQo+ID4gdHJv bmRteUBoYW1tZXJzcGFjZS5jb20+IHdyb3RlOg0KPiA+IA0KPiA+IE9uIFdlZCwgMjAxOC0wOS0w NSBhdCAxMDozMCAtMDQwMCwgQ2h1Y2sgTGV2ZXIgd3JvdGU6DQo+ID4gPiA+IE9uIFNlcCA0LCAy MDE4LCBhdCA1OjA1IFBNLCBUcm9uZCBNeWtsZWJ1c3QgPHRyb25kbXlAZ21haWwuY29tPg0KPiA+ ID4gPiB3cm90ZToNCj4gPiA+ID4gDQo+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IFRyb25kIE15a2xl YnVzdCA8dHJvbmQubXlrbGVidXN0QGhhbW1lcnNwYWNlLmNvbQ0KPiA+ID4gPiA+DQo+ID4gPiA+ IC0tLQ0KPiA+ID4gPiBuZXQvc3VucnBjL2NsbnQuYyB8ICA2IC0tLS0tLQ0KPiA+ID4gPiBuZXQv c3VucnBjL3hwcnQuYyB8IDEzICsrKysrKy0tLS0tLS0NCj4gPiA+ID4gMiBmaWxlcyBjaGFuZ2Vk LCA2IGluc2VydGlvbnMoKyksIDEzIGRlbGV0aW9ucygtKQ0KPiA+ID4gPiANCj4gPiA+ID4gZGlm ZiAtLWdpdCBhL25ldC9zdW5ycGMvY2xudC5jIGIvbmV0L3N1bnJwYy9jbG50LmMNCj4gPiA+ID4g aW5kZXggMDMyYjcwNDJhZGI2Li4yMTcxM2JlZDgxMmEgMTAwNjQ0DQo+ID4gPiA+IC0tLSBhL25l dC9zdW5ycGMvY2xudC5jDQo+ID4gPiA+ICsrKyBiL25ldC9zdW5ycGMvY2xudC5jDQo+ID4gPiA+ IEBAIC0xOTY3LDggKzE5NjcsNiBAQCBjYWxsX2Nvbm5lY3Rfc3RhdHVzKHN0cnVjdCBycGNfdGFz aw0KPiA+ID4gPiAqdGFzaykNCj4gPiA+ID4gc3RhdGljIHZvaWQNCj4gPiA+ID4gY2FsbF90cmFu c21pdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ID4gPiA+IHsNCj4gPiA+ID4gLQlpbnQgaXNf cmV0cmFucyA9IFJQQ19XQVNfU0VOVCh0YXNrKTsNCj4gPiA+ID4gLQ0KPiA+ID4gPiAJZHByaW50 X3N0YXR1cyh0YXNrKTsNCj4gPiA+ID4gDQo+ID4gPiA+IAl0YXNrLT50a19hY3Rpb24gPSBjYWxs X3RyYW5zbWl0X3N0YXR1czsNCj4gPiA+ID4gQEAgLTE5NzksMTAgKzE5NzcsNiBAQCBjYWxsX3Ry YW5zbWl0KHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4gPiA+ID4gCWlmICgheHBydF9wcmVwYXJl X3RyYW5zbWl0KHRhc2spKQ0KPiA+ID4gPiAJCXJldHVybjsNCj4gPiA+ID4gCXhwcnRfdHJhbnNt aXQodGFzayk7DQo+ID4gPiA+IC0JaWYgKHRhc2stPnRrX3N0YXR1cyA8IDApDQo+ID4gPiA+IC0J CXJldHVybjsNCj4gPiA+ID4gLQlpZiAoaXNfcmV0cmFucykNCj4gPiA+ID4gLQkJdGFzay0+dGtf Y2xpZW50LT5jbF9zdGF0cy0+cnBjcmV0cmFucysrOw0KPiA+ID4gPiB9DQo+ID4gPiA+IA0KPiA+ ID4gPiAvKg0KPiA+ID4gPiBkaWZmIC0tZ2l0IGEvbmV0L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3Vu cnBjL3hwcnQuYw0KPiA+ID4gPiBpbmRleCBmMTMwMWQzOTEzOTkuLmUyZjViNDY2OGE2NiAxMDA2 NDQNCj4gPiA+ID4gLS0tIGEvbmV0L3N1bnJwYy94cHJ0LmMNCj4gPiA+ID4gKysrIGIvbmV0L3N1 bnJwYy94cHJ0LmMNCj4gPiA+ID4gQEAgLTE5MSw4ICsxOTEsNiBAQCBpbnQgeHBydF9yZXNlcnZl X3hwcnQoc3RydWN0IHJwY194cHJ0DQo+ID4gPiA+ICp4cHJ0LA0KPiA+ID4gPiBzdHJ1Y3QgcnBj X3Rhc2sgKnRhc2spDQo+ID4gPiA+IAkJZ290byBvdXRfc2xlZXA7DQo+ID4gPiA+IAl9DQo+ID4g PiA+IAl4cHJ0LT5zbmRfdGFzayA9IHRhc2s7DQo+ID4gPiA+IC0JaWYgKHJlcSAhPSBOVUxMKQ0K PiA+ID4gPiAtCQlyZXEtPnJxX250cmFucysrOw0KPiA+ID4gPiANCj4gPiA+ID4gCXJldHVybiAx Ow0KPiA+ID4gPiANCj4gPiA+ID4gQEAgLTI0Nyw3ICsyNDUsNiBAQCBpbnQgeHBydF9yZXNlcnZl X3hwcnRfY29uZyhzdHJ1Y3QgcnBjX3hwcnQNCj4gPiA+ID4gKnhwcnQsIHN0cnVjdCBycGNfdGFz ayAqdGFzaykNCj4gPiA+ID4gCX0NCj4gPiA+ID4gCWlmIChfX3hwcnRfZ2V0X2NvbmcoeHBydCwg dGFzaykpIHsNCj4gPiA+ID4gCQl4cHJ0LT5zbmRfdGFzayA9IHRhc2s7DQo+ID4gPiA+IC0JCXJl cS0+cnFfbnRyYW5zKys7DQo+ID4gPiA+IAkJcmV0dXJuIDE7DQo+ID4gPiA+IAl9DQo+ID4gPiA+ IAl4cHJ0X2NsZWFyX2xvY2tlZCh4cHJ0KTsNCj4gPiA+ID4gQEAgLTI4MSwxMiArMjc4LDggQEAg c3RhdGljIGlubGluZSBpbnQgeHBydF9sb2NrX3dyaXRlKHN0cnVjdA0KPiA+ID4gPiBycGNfeHBy dCAqeHBydCwgc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+ID4gPiBzdGF0aWMgYm9vbCBfX3hw cnRfbG9ja193cml0ZV9mdW5jKHN0cnVjdCBycGNfdGFzayAqdGFzaywgdm9pZA0KPiA+ID4gPiAq ZGF0YSkNCj4gPiA+ID4gew0KPiA+ID4gPiAJc3RydWN0IHJwY194cHJ0ICp4cHJ0ID0gZGF0YTsN Cj4gPiA+ID4gLQlzdHJ1Y3QgcnBjX3Jxc3QgKnJlcTsNCj4gPiA+ID4gDQo+ID4gPiA+IC0JcmVx ID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gPiA+IAl4cHJ0LT5zbmRfdGFzayA9IHRhc2s7DQo+ID4g PiA+IC0JaWYgKHJlcSkNCj4gPiA+ID4gLQkJcmVxLT5ycV9udHJhbnMrKzsNCj4gPiA+ID4gCXJl dHVybiB0cnVlOw0KPiA+ID4gPiB9DQo+ID4gPiA+IA0KPiA+ID4gPiBAQCAtMTEyNiw2ICsxMTE5 LDcgQEAgdm9pZCB4cHJ0X3RyYW5zbWl0KHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4gPiA+ID4g CXN0cnVjdCBycGNfcnFzdAkqcmVxID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gPiA+IAlzdHJ1Y3Qg cnBjX3hwcnQJKnhwcnQgPSByZXEtPnJxX3hwcnQ7DQo+ID4gPiA+IAl1bnNpZ25lZCBpbnQgY29u bmVjdF9jb29raWU7DQo+ID4gPiA+ICsJaW50IGlzX3JldHJhbnMgPSBSUENfV0FTX1NFTlQodGFz ayk7DQo+ID4gPiA+IAlpbnQgc3RhdHVzOw0KPiA+ID4gPiANCj4gPiA+ID4gCWRwcmludGsoIlJQ QzogJTV1IHhwcnRfdHJhbnNtaXQoJXUpXG4iLCB0YXNrLT50a19waWQsDQo+ID4gPiA+IHJlcS0N Cj4gPiA+ID4gPiBycV9zbGVuKTsNCj4gPiA+ID4gDQo+ID4gPiA+IEBAIC0xMTQwLDYgKzExMzQs OCBAQCB2b2lkIHhwcnRfdHJhbnNtaXQoc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiA+ID4gPiAJ CX0NCj4gPiA+ID4gCX0NCj4gPiA+ID4gDQo+ID4gPiA+ICsJcmVxLT5ycV9udHJhbnMrKzsNCj4g PiA+ID4gKw0KPiA+ID4gDQo+ID4gPiBycV9udHJhbnMgaXMgdXNlZCBpbiB0d28gcGxhY2VzOg0K PiA+ID4gDQo+ID4gPiAxLiBycGNfdXBkYXRlX3J0dA0KPiA+ID4gMi4gcnBjX2NvdW50X2lvc3Rh dHNfbWV0cmljcw0KPiA+ID4gDQo+ID4gPiBCb3RoIG9mIHRoZXNlIGFwcGVhciB0byBhc3N1bWUg dGhhdCBycV9udHJhbnMgaXMgY291bnRpbmcNCj4gPiA+IGZ1bGwgc3VjY2Vzc2Z1bCB0cmFuc21p c3Npb25zIChpZSwgY2FsbHMgdGhhdCBhcHBlYXIgb24gdGhlDQo+ID4gPiB3aXJlKSwgbm90IGlu dm9jYXRpb25zIG9mIC0+c2VuZF9yZXF1ZXN0KCkuDQo+ID4gPiANCj4gPiA+IENhbiB0aGlzIGNv dW50ZXIgYmUgbW92ZWQgZG93biB0byB3aGVyZSBycGNyZXRyYW5zIGlzIHVwZGF0ZWQ/DQo+ID4g PiANCj4gPiANCj4gPiBUaGUgcmVhc29uIHdoeSBJIGRvbid0IHdhbnQgdG8gdXBkYXRlIHJlcS0+ cnFfbnRyYW5zIGFmdGVyDQo+ID4gdHJhbnNtaXNzaW9uDQo+ID4gaXMgYmVjYXVzZSB0aGF0IHdv dWxkIHJhY2Ugd2l0aCB0aGUgcmVwbHkgZnJvbSB0aGUgc2VydmVyICh3aGljaCBpcw0KPiA+IG5v dA0KPiA+IGJsb2NrZWQgYnkgdGhlIFhQUlRfTE9DSykuIEluIHRoYXQgcmFjZSBzY2VuYXJpbywg cnBjX3VwZGF0ZV9ydHQoKQ0KPiA+IGNvdWxkIHVwZGF0ZSB0aGUgUlRUIHZhbHVlIGJlZm9yZSB3 ZSd2ZSB1cGRhdGVkIHRoZSByZXEtPnJxX250cmFucywNCj4gPiBtZWFuaW5nIHRoYXQgd2UgbWln aHQgYmUgbWVhc3VyaW5nIHRoZSByZXNwb25zZSB0aW1lIGFnYWluc3QgYQ0KPiA+IHJlcGx5IHRv DQo+ID4gdGhlIG9yaWdpbmFsIHRyYW5zbWlzc2lvbiBvciB0byB0aGUgcmV0cmFuc21pc3Npb24g dGhhdCB3YXMganVzdA0KPiA+IHNlbnQuDQo+ID4gVGhhdCB3b3VsZCBlbmQgdXAgYnJlYWtpbmcg VmFuIEphY29ic2VuIGNvbmdlc3Rpb24gY29udHJvbCwgc2luY2UNCj4gPiB3ZQ0KPiA+IHJpc2sg c2lnbmlmaWNhbnRseSB1bmRlcmVzdGltYXRpbmcgdGhlIFJUVCB2YWx1ZS4NCj4gDQo+IEJ1dCBm b3IgdHJhbnNwb3J0cyB0aGF0IGRvbid0IHVzZSB4cHJ0X3VwZGF0ZV9ydHQsIGVycm9yDQo+IHJl dHVybnMgZnJvbSAtPnNlbmRfcmVxdWVzdCgpIChsaWtlIC1FQUdBSU4gb3IgLUVOT0JVRlMpDQo+ IG1lYW5zIHRoYXQgcnFfbnRyYW5zIGlzIGNvdW50ZWQgdHdpY2UsIGFuZCBpdCBzaG93cyB1cCBh cw0KPiBmYWxzZSByZXRyYW5zbWlzc2lvbnMuDQoNClRoYXQncyB0aGUgd2F5IGl0IGhhcyBhbHdh eXMgd29ya2VkLiBUaGUgVmFuIEphY29ic2VuIGNvZGUgd2FzDQppbXBsZW1lbnRlZCBsb25nIGJl Zm9yZSB0aGUgcGVyLXJlcXVlc3Qgc3RhdHMgd2VyZSBpbnRyb2R1Y2VkLCBhbmQgc28NCnRoZSBi ZWhhdmlvdXIgb2YgcmVxLT5ycV9udHJhbnMgaGFzIGFsd2F5cyBiZWVuIHRvIHVwZGF0ZSB0aGUg dmFsdWUNCmJlZm9yZSB3ZSB0cmFuc21pdC4NCg0KPiBXaGF0IGlmIHJxX250cmFucyB3YXMgYnVt cGVkIGluIHRoZSB0cmFuc3BvcnQgY29kZSBpbnN0ZWFkPw0KDQpJZiB3ZSBtdXN0IGNoYW5nZSB0 aGUgYmVoYXZpb3VyLCB0aGVuIHdoeSBub3QganVzdCBoYXZlIHRoZSBnZW5lcmljDQpsYXllciAn dW5idW1wJyB0aGUgY291bnRlciBvbiBhbiBpbmNvbXBsZXRlIHRyYW5zbWlzc2lvbj8NCg0KLS0g DQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lciwgSGFtbWVyc3Bh Y2UNCnRyb25kLm15a2xlYnVzdEBoYW1tZXJzcGFjZS5jb20NCg0KDQo= ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() 2018-09-05 16:07 ` Trond Myklebust @ 2018-09-05 16:34 ` Chuck Lever 0 siblings, 0 replies; 45+ messages in thread From: Chuck Lever @ 2018-09-05 16:34 UTC (permalink / raw) To: Trond Myklebust; +Cc: Linux NFS Mailing List > On Sep 5, 2018, at 12:07 PM, Trond Myklebust <trondmy@hammerspace.com> = wrote: >=20 > On Wed, 2018-09-05 at 11:31 -0400, Chuck Lever wrote: >>> On Sep 5, 2018, at 11:28 AM, Trond Myklebust < >>> trondmy@hammerspace.com> wrote: >>>=20 >>> On Wed, 2018-09-05 at 10:30 -0400, Chuck Lever wrote: >>>>> On Sep 4, 2018, at 5:05 PM, Trond Myklebust <trondmy@gmail.com> >>>>> wrote: >>>>>=20 >>>>> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com >>>>>>=20 >>>>> --- >>>>> net/sunrpc/clnt.c | 6 ------ >>>>> net/sunrpc/xprt.c | 13 ++++++------- >>>>> 2 files changed, 6 insertions(+), 13 deletions(-) >>>>>=20 >>>>> diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c >>>>> index 032b7042adb6..21713bed812a 100644 >>>>> --- a/net/sunrpc/clnt.c >>>>> +++ b/net/sunrpc/clnt.c >>>>> @@ -1967,8 +1967,6 @@ call_connect_status(struct rpc_task >>>>> *task) >>>>> static void >>>>> call_transmit(struct rpc_task *task) >>>>> { >>>>> - int is_retrans =3D RPC_WAS_SENT(task); >>>>> - >>>>> dprint_status(task); >>>>>=20 >>>>> task->tk_action =3D call_transmit_status; >>>>> @@ -1979,10 +1977,6 @@ call_transmit(struct rpc_task *task) >>>>> if (!xprt_prepare_transmit(task)) >>>>> return; >>>>> xprt_transmit(task); >>>>> - if (task->tk_status < 0) >>>>> - return; >>>>> - if (is_retrans) >>>>> - task->tk_client->cl_stats->rpcretrans++; >>>>> } >>>>>=20 >>>>> /* >>>>> diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c >>>>> index f1301d391399..e2f5b4668a66 100644 >>>>> --- a/net/sunrpc/xprt.c >>>>> +++ b/net/sunrpc/xprt.c >>>>> @@ -191,8 +191,6 @@ int xprt_reserve_xprt(struct rpc_xprt >>>>> *xprt, >>>>> struct rpc_task *task) >>>>> goto out_sleep; >>>>> } >>>>> xprt->snd_task =3D task; >>>>> - if (req !=3D NULL) >>>>> - req->rq_ntrans++; >>>>>=20 >>>>> return 1; >>>>>=20 >>>>> @@ -247,7 +245,6 @@ int xprt_reserve_xprt_cong(struct rpc_xprt >>>>> *xprt, struct rpc_task *task) >>>>> } >>>>> if (__xprt_get_cong(xprt, task)) { >>>>> xprt->snd_task =3D task; >>>>> - req->rq_ntrans++; >>>>> return 1; >>>>> } >>>>> xprt_clear_locked(xprt); >>>>> @@ -281,12 +278,8 @@ static inline int xprt_lock_write(struct >>>>> rpc_xprt *xprt, struct rpc_task *task) >>>>> static bool __xprt_lock_write_func(struct rpc_task *task, void >>>>> *data) >>>>> { >>>>> struct rpc_xprt *xprt =3D data; >>>>> - struct rpc_rqst *req; >>>>>=20 >>>>> - req =3D task->tk_rqstp; >>>>> xprt->snd_task =3D task; >>>>> - if (req) >>>>> - req->rq_ntrans++; >>>>> return true; >>>>> } >>>>>=20 >>>>> @@ -1126,6 +1119,7 @@ void xprt_transmit(struct rpc_task *task) >>>>> struct rpc_rqst *req =3D task->tk_rqstp; >>>>> struct rpc_xprt *xprt =3D req->rq_xprt; >>>>> unsigned int connect_cookie; >>>>> + int is_retrans =3D RPC_WAS_SENT(task); >>>>> int status; >>>>>=20 >>>>> dprintk("RPC: %5u xprt_transmit(%u)\n", task->tk_pid, >>>>> req- >>>>>> rq_slen); >>>>>=20 >>>>> @@ -1140,6 +1134,8 @@ void xprt_transmit(struct rpc_task *task) >>>>> } >>>>> } >>>>>=20 >>>>> + req->rq_ntrans++; >>>>> + >>>>=20 >>>> rq_ntrans is used in two places: >>>>=20 >>>> 1. rpc_update_rtt >>>> 2. rpc_count_iostats_metrics >>>>=20 >>>> Both of these appear to assume that rq_ntrans is counting >>>> full successful transmissions (ie, calls that appear on the >>>> wire), not invocations of ->send_request(). >>>>=20 >>>> Can this counter be moved down to where rpcretrans is updated? >>>>=20 >>>=20 >>> The reason why I don't want to update req->rq_ntrans after >>> transmission >>> is because that would race with the reply from the server (which is >>> not >>> blocked by the XPRT_LOCK). In that race scenario, rpc_update_rtt() >>> could update the RTT value before we've updated the req->rq_ntrans, >>> meaning that we might be measuring the response time against a >>> reply to >>> the original transmission or to the retransmission that was just >>> sent. >>> That would end up breaking Van Jacobsen congestion control, since >>> we >>> risk significantly underestimating the RTT value. >>=20 >> But for transports that don't use xprt_update_rtt, error >> returns from ->send_request() (like -EAGAIN or -ENOBUFS) >> means that rq_ntrans is counted twice, and it shows up as >> false retransmissions. >=20 > That's the way it has always worked. Not quite: the current xprt_reserve_xprt_cong has a latch -- rq_cong. req->rq_ntrans is bumped only once when it gets a congestion credit. The transport send lock is held until enough ->send_request calls have been done to transmit the whole Call message, but rq_ntrans is bumped only once per full RPC Call message. Note that this code is a little dodgy for xprt_reserve_xprt: when driving a connect, the RPC task takes and releases the transport send lock twice, and both times it bumps rq_ntrans, which is a bug. This patch appears to fix that. > The Van Jacobsen code was > implemented long before the per-request stats were introduced, and so > the behaviour of req->rq_ntrans has always been to update the value > before we transmit. >> What if rq_ntrans was bumped in the transport code instead? >=20 > If we must change the behaviour, then why not just have the generic > layer 'unbump' the counter on an incomplete transmission? OK. There's no way for a reply to arrive if the call hasn't been fully transmitted, so a race is avoided in that case. -- Chuck Lever ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() 2018-09-04 21:05 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 21/34] SUNRPC: Don't reset the request 'bytes_sent' counter when releasing XPRT_LOCK Trond Myklebust @ 2018-09-06 18:49 ` Schumaker, Anna 2018-09-06 18:57 ` Trond Myklebust 1 sibling, 1 reply; 45+ messages in thread From: Schumaker, Anna @ 2018-09-06 18:49 UTC (permalink / raw) To: linux-nfs, trondmy SGkgVHJvbmQsDQoNCk9uIFR1ZSwgMjAxOC0wOS0wNCBhdCAxNzowNSAtMDQwMCwgVHJvbmQgTXlr bGVidXN0IHdyb3RlOg0KPiBXaGVuIHdlIHNoaWZ0IHRvIHVzaW5nIHRoZSB0cmFuc21pdCBxdWV1 ZSwgdGhlbiB0aGUgdGFzayB0aGF0IGhvbGRzIHRoZQ0KPiB3cml0ZSBsb2NrIHdpbGwgbm90IG5l Y2Vzc2FyaWx5IGJlIHRoZSBzYW1lIGFzIHRoZSBvbmUgYmVpbmcgdHJhbnNtaXR0ZWQuDQo+IA0K PiBTaWduZWQtb2ZmLWJ5OiBUcm9uZCBNeWtsZWJ1c3QgPHRyb25kLm15a2xlYnVzdEBoYW1tZXJz cGFjZS5jb20+DQo+IC0tLQ0KPiAgaW5jbHVkZS9saW51eC9zdW5ycGMveHBydC5oICAgICAgICAg ICAgICAgIHwgIDIgKy0NCj4gIG5ldC9zdW5ycGMveHBydC5jICAgICAgICAgICAgICAgICAgICAg ICAgICB8ICAyICstDQo+ICBuZXQvc3VucnBjL3hwcnRyZG1hL3N2Y19yZG1hX2JhY2tjaGFubmVs LmMgfCAgMyArLS0NCj4gIG5ldC9zdW5ycGMveHBydHJkbWEvdHJhbnNwb3J0LmMgICAgICAgICAg ICB8ICA1ICsrLS0NCj4gIG5ldC9zdW5ycGMveHBydHNvY2suYyAgICAgICAgICAgICAgICAgICAg ICB8IDI3ICsrKysrKysrKysrLS0tLS0tLS0tLS0NCj4gIDUgZmlsZXMgY2hhbmdlZCwgMTggaW5z ZXJ0aW9ucygrKSwgMjEgZGVsZXRpb25zKC0pDQo+IA0KPiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS9s aW51eC9zdW5ycGMveHBydC5oIGIvaW5jbHVkZS9saW51eC9zdW5ycGMveHBydC5oDQo+IGluZGV4 IDgxYTZjMmM4ZGZjNy4uNmQ5MWFjZmUwNjQ0IDEwMDY0NA0KPiAtLS0gYS9pbmNsdWRlL2xpbnV4 L3N1bnJwYy94cHJ0LmgNCj4gKysrIGIvaW5jbHVkZS9saW51eC9zdW5ycGMveHBydC5oDQo+IEBA IC0xNDAsNyArMTQwLDcgQEAgc3RydWN0IHJwY194cHJ0X29wcyB7DQo+ICAJdm9pZAkJKCpjb25u ZWN0KShzdHJ1Y3QgcnBjX3hwcnQgKnhwcnQsIHN0cnVjdCBycGNfdGFzaw0KPiAqdGFzayk7DQo+ ICAJaW50CQkoKmJ1Zl9hbGxvYykoc3RydWN0IHJwY190YXNrICp0YXNrKTsNCj4gIAl2b2lkCQko KmJ1Zl9mcmVlKShzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spOw0KPiAtCWludAkJKCpzZW5kX3JlcXVl c3QpKHN0cnVjdCBycGNfdGFzayAqdGFzayk7DQo+ICsJaW50CQkoKnNlbmRfcmVxdWVzdCkoc3Ry dWN0IHJwY19ycXN0ICpyZXEsIHN0cnVjdCBycGNfdGFzaw0KPiAqdGFzayk7DQo+ICAJdm9pZAkJ KCpzZXRfcmV0cmFuc190aW1lb3V0KShzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spOw0KPiAgCXZvaWQJ CSgqdGltZXIpKHN0cnVjdCBycGNfeHBydCAqeHBydCwgc3RydWN0IHJwY190YXNrDQo+ICp0YXNr KTsNCj4gIAl2b2lkCQkoKnJlbGVhc2VfcmVxdWVzdCkoc3RydWN0IHJwY190YXNrICp0YXNrKTsN Cj4gZGlmZiAtLWdpdCBhL25ldC9zdW5ycGMveHBydC5jIGIvbmV0L3N1bnJwYy94cHJ0LmMNCj4g aW5kZXggOGU4YzM0NWVlZGY3Li43YzgwZjkzNTYyZTUgMTAwNjQ0DQo+IC0tLSBhL25ldC9zdW5y cGMveHBydC5jDQo+ICsrKyBiL25ldC9zdW5ycGMveHBydC5jDQo+IEBAIC0xMTcwLDcgKzExNzAs NyBAQCB2b2lkIHhwcnRfdHJhbnNtaXQoc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiAgCX0NCj4g IA0KPiAgCWNvbm5lY3RfY29va2llID0geHBydC0+Y29ubmVjdF9jb29raWU7DQo+IC0Jc3RhdHVz ID0geHBydC0+b3BzLT5zZW5kX3JlcXVlc3QodGFzayk7DQo+ICsJc3RhdHVzID0geHBydC0+b3Bz LT5zZW5kX3JlcXVlc3QocmVxLCB0YXNrKTsNCj4gIAl0cmFjZV94cHJ0X3RyYW5zbWl0KHhwcnQs IHJlcS0+cnFfeGlkLCBzdGF0dXMpOw0KPiAgCWlmIChzdGF0dXMgIT0gMCkgew0KPiAgCQl0YXNr LT50a19zdGF0dXMgPSBzdGF0dXM7DQo+IGRpZmYgLS1naXQgYS9uZXQvc3VucnBjL3hwcnRyZG1h L3N2Y19yZG1hX2JhY2tjaGFubmVsLmMNCj4gYi9uZXQvc3VucnBjL3hwcnRyZG1hL3N2Y19yZG1h X2JhY2tjaGFubmVsLmMNCj4gaW5kZXggMDliMTJiNzU2OGZlLi5kMTYxOGM3MGVkYjQgMTAwNjQ0 DQo+IC0tLSBhL25ldC9zdW5ycGMveHBydHJkbWEvc3ZjX3JkbWFfYmFja2NoYW5uZWwuYw0KPiAr KysgYi9uZXQvc3VucnBjL3hwcnRyZG1hL3N2Y19yZG1hX2JhY2tjaGFubmVsLmMNCj4gQEAgLTIx NSw5ICsyMTUsOCBAQCBycGNyZG1hX2JjX3NlbmRfcmVxdWVzdChzdHJ1Y3Qgc3ZjeHBydF9yZG1h ICpyZG1hLCBzdHJ1Y3QNCj4gcnBjX3Jxc3QgKnJxc3QpDQo+ICAgKiBjb25uZWN0aW9uLg0KPiAg ICovDQo+ICBzdGF0aWMgaW50DQo+IC14cHJ0X3JkbWFfYmNfc2VuZF9yZXF1ZXN0KHN0cnVjdCBy cGNfdGFzayAqdGFzaykNCj4gK3hwcnRfcmRtYV9iY19zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY19y cXN0ICpycXN0LCBzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ICB7DQo+IC0Jc3RydWN0IHJwY19y cXN0ICpycXN0ID0gdGFzay0+dGtfcnFzdHA7DQo+ICAJc3RydWN0IHN2Y194cHJ0ICpzeHBydCA9 IHJxc3QtPnJxX3hwcnQtPmJjX3hwcnQ7DQo+ICAJc3RydWN0IHN2Y3hwcnRfcmRtYSAqcmRtYTsN Cj4gIAlpbnQgcmV0Ow0KPiBkaWZmIC0tZ2l0IGEvbmV0L3N1bnJwYy94cHJ0cmRtYS90cmFuc3Bv cnQuYyBiL25ldC9zdW5ycGMveHBydHJkbWEvdHJhbnNwb3J0LmMNCj4gaW5kZXggMTQzY2UyNTc5 YmE5Li5mYTY4NGJmNGQwOTAgMTAwNjQ0DQo+IC0tLSBhL25ldC9zdW5ycGMveHBydHJkbWEvdHJh bnNwb3J0LmMNCj4gKysrIGIvbmV0L3N1bnJwYy94cHJ0cmRtYS90cmFuc3BvcnQuYw0KPiBAQCAt NzA2LDkgKzcwNiw4IEBAIHhwcnRfcmRtYV9mcmVlKHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4g ICAqCQlzZW50LiBEbyBub3QgdHJ5IHRvIHNlbmQgdGhpcyBtZXNzYWdlIGFnYWluLg0KPiAgICov DQo+ICBzdGF0aWMgaW50DQo+IC14cHJ0X3JkbWFfc2VuZF9yZXF1ZXN0KHN0cnVjdCBycGNfdGFz ayAqdGFzaykNCj4gK3hwcnRfcmRtYV9zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY19ycXN0ICpycXN0 LCBzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ICB7DQo+IC0Jc3RydWN0IHJwY19ycXN0ICpycXN0 ID0gdGFzay0+dGtfcnFzdHA7DQo+ICAJc3RydWN0IHJwY194cHJ0ICp4cHJ0ID0gcnFzdC0+cnFf eHBydDsNCj4gIAlzdHJ1Y3QgcnBjcmRtYV9yZXEgKnJlcSA9IHJwY3JfdG9fcmRtYXIocnFzdCk7 DQo+ICAJc3RydWN0IHJwY3JkbWFfeHBydCAqcl94cHJ0ID0gcnBjeF90b19yZG1heCh4cHJ0KTsN Cj4gQEAgLTc0MSw3ICs3NDAsNyBAQCB4cHJ0X3JkbWFfc2VuZF9yZXF1ZXN0KHN0cnVjdCBycGNf dGFzayAqdGFzaykNCj4gIAkvKiBBbiBSUEMgd2l0aCBubyByZXBseSB3aWxsIHRocm93IG9mZiBj cmVkaXQgYWNjb3VudGluZywNCj4gIAkgKiBzbyBkcm9wIHRoZSBjb25uZWN0aW9uIHRvIHJlc2V0 IHRoZSBjcmVkaXQgZ3JhbnQuDQo+ICAJICovDQo+IC0JaWYgKCFycGNfcmVwbHlfZXhwZWN0ZWQo dGFzaykpDQo+ICsJaWYgKCFycGNfcmVwbHlfZXhwZWN0ZWQocnFzdC0+cnFfdGFzaykpDQo+ICAJ CWdvdG8gZHJvcF9jb25uZWN0aW9uOw0KPiAgCXJldHVybiAwOw0KPiAgDQo+IGRpZmYgLS1naXQg YS9uZXQvc3VucnBjL3hwcnRzb2NrLmMgYi9uZXQvc3VucnBjL3hwcnRzb2NrLmMNCj4gaW5kZXgg OGQ2NDA0MjU5ZmY5Li5iODE0M2VkZWQ0YWYgMTAwNjQ0DQo+IC0tLSBhL25ldC9zdW5ycGMveHBy dHNvY2suYw0KPiArKysgYi9uZXQvc3VucnBjL3hwcnRzb2NrLmMNCj4gQEAgLTQ0OSwxMiArNDQ5 LDEyIEBAIHN0YXRpYyB2b2lkIHhzX25vc3BhY2VfY2FsbGJhY2soc3RydWN0IHJwY190YXNrICp0 YXNrKQ0KPiAgDQo+ICAvKioNCj4gICAqIHhzX25vc3BhY2UgLSBwbGFjZSB0YXNrIG9uIHdhaXQg cXVldWUgaWYgdHJhbnNtaXQgd2FzIGluY29tcGxldGUNCj4gKyAqIEByZXE6IHBvaW50ZXIgdG8g UlBDIHJlcXVlc3QNCj4gICAqIEB0YXNrOiB0YXNrIHRvIHB1dCB0byBzbGVlcA0KPiAgICoNCj4g ICAqLw0KPiAtc3RhdGljIGludCB4c19ub3NwYWNlKHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4g K3N0YXRpYyBpbnQgeHNfbm9zcGFjZShzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSwgc3RydWN0IHJwY190 YXNrICp0YXNrKQ0KPiAgew0KPiAtCXN0cnVjdCBycGNfcnFzdCAqcmVxID0gdGFzay0+dGtfcnFz dHA7DQo+ICAJc3RydWN0IHJwY194cHJ0ICp4cHJ0ID0gcmVxLT5ycV94cHJ0Ow0KPiAgCXN0cnVj dCBzb2NrX3hwcnQgKnRyYW5zcG9ydCA9IGNvbnRhaW5lcl9vZih4cHJ0LCBzdHJ1Y3Qgc29ja194 cHJ0LA0KPiB4cHJ0KTsNCj4gIAlzdHJ1Y3Qgc29jayAqc2sgPSB0cmFuc3BvcnQtPmluZXQ7DQo+ IEBAIC01MTMsNiArNTEzLDcgQEAgc3RhdGljIGlubGluZSB2b2lkIHhzX2VuY29kZV9zdHJlYW1f cmVjb3JkX21hcmtlcihzdHJ1Y3QNCj4geGRyX2J1ZiAqYnVmKQ0KPiAgDQo+ICAvKioNCj4gICAq IHhzX2xvY2FsX3NlbmRfcmVxdWVzdCAtIHdyaXRlIGFuIFJQQyByZXF1ZXN0IHRvIGFuIEFGX0xP Q0FMIHNvY2tldA0KPiArICogQHJlcTogcG9pbnRlciB0byBSUEMgcmVxdWVzdA0KPiAgICogQHRh c2s6IFJQQyB0YXNrIHRoYXQgbWFuYWdlcyB0aGUgc3RhdGUgb2YgYW4gUlBDIHJlcXVlc3QNCj4g ICAqDQo+ICAgKiBSZXR1cm4gdmFsdWVzOg0KPiBAQCAtNTIyLDkgKzUyMyw4IEBAIHN0YXRpYyBp bmxpbmUgdm9pZCB4c19lbmNvZGVfc3RyZWFtX3JlY29yZF9tYXJrZXIoc3RydWN0DQo+IHhkcl9i dWYgKmJ1ZikNCj4gICAqIEVOT1RDT05OOglDYWxsZXIgbmVlZHMgdG8gaW52b2tlIGNvbm5lY3Qg bG9naWMgdGhlbiBjYWxsIGFnYWluDQo+ICAgKiAgICBvdGhlcjoJU29tZSBvdGhlciBlcnJvciBv Y2N1cmVkLCB0aGUgcmVxdWVzdCB3YXMgbm90IHNlbnQNCj4gICAqLw0KPiAtc3RhdGljIGludCB4 c19sb2NhbF9zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiArc3RhdGljIGlu dCB4c19sb2NhbF9zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY19ycXN0ICpyZXEsIHN0cnVjdCBycGNf dGFzayAqdGFzaykNCj4gIHsNCj4gLQlzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSA9IHRhc2stPnRrX3Jx c3RwOw0KPiAgCXN0cnVjdCBycGNfeHBydCAqeHBydCA9IHJlcS0+cnFfeHBydDsNCj4gIAlzdHJ1 Y3Qgc29ja194cHJ0ICp0cmFuc3BvcnQgPQ0KPiAgCQkJCWNvbnRhaW5lcl9vZih4cHJ0LCBzdHJ1 Y3Qgc29ja194cHJ0LCB4cHJ0KTsNCj4gQEAgLTU2OSw3ICs1NjksNyBAQCBzdGF0aWMgaW50IHhz X2xvY2FsX3NlbmRfcmVxdWVzdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ICAJY2FzZSAtRU5P QlVGUzoNCj4gIAkJYnJlYWs7DQo+ICAJY2FzZSAtRUFHQUlOOg0KPiAtCQlzdGF0dXMgPSB4c19u b3NwYWNlKHRhc2spOw0KPiArCQlzdGF0dXMgPSB4c19ub3NwYWNlKHJlcSwgdGFzayk7DQo+ICAJ CWJyZWFrOw0KPiAgCWRlZmF1bHQ6DQo+ICAJCWRwcmludGsoIlJQQzogICAgICAgc2VuZG1zZyBy ZXR1cm5lZCB1bnJlY29nbml6ZWQgZXJyb3IgJWRcbiIsDQo+IEBAIC01ODUsNiArNTg1LDcgQEAg c3RhdGljIGludCB4c19sb2NhbF9zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY190YXNrICp0YXNrKQ0K PiAgDQo+ICAvKioNCj4gICAqIHhzX3VkcF9zZW5kX3JlcXVlc3QgLSB3cml0ZSBhbiBSUEMgcmVx dWVzdCB0byBhIFVEUCBzb2NrZXQNCj4gKyAqIEByZXE6IHBvaW50ZXIgdG8gUlBDIHJlcXVlc3QN Cj4gICAqIEB0YXNrOiBhZGRyZXNzIG9mIFJQQyB0YXNrIHRoYXQgbWFuYWdlcyB0aGUgc3RhdGUg b2YgYW4gUlBDIHJlcXVlc3QNCj4gICAqDQo+ICAgKiBSZXR1cm4gdmFsdWVzOg0KPiBAQCAtNTk0 LDkgKzU5NSw4IEBAIHN0YXRpYyBpbnQgeHNfbG9jYWxfc2VuZF9yZXF1ZXN0KHN0cnVjdCBycGNf dGFzayAqdGFzaykNCj4gICAqIEVOT1RDT05OOglDYWxsZXIgbmVlZHMgdG8gaW52b2tlIGNvbm5l Y3QgbG9naWMgdGhlbiBjYWxsIGFnYWluDQo+ICAgKiAgICBvdGhlcjoJU29tZSBvdGhlciBlcnJv ciBvY2N1cnJlZCwgdGhlIHJlcXVlc3Qgd2FzIG5vdCBzZW50DQo+ICAgKi8NCj4gLXN0YXRpYyBp bnQgeHNfdWRwX3NlbmRfcmVxdWVzdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ICtzdGF0aWMg aW50IHhzX3VkcF9zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY19ycXN0ICpyZXEsIHN0cnVjdCBycGNf dGFzayAqdGFzaykNCj4gIHsNCj4gLQlzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSA9IHRhc2stPnRrX3Jx c3RwOw0KPiAgCXN0cnVjdCBycGNfeHBydCAqeHBydCA9IHJlcS0+cnFfeHBydDsNCj4gIAlzdHJ1 Y3Qgc29ja194cHJ0ICp0cmFuc3BvcnQgPSBjb250YWluZXJfb2YoeHBydCwgc3RydWN0IHNvY2tf eHBydCwNCj4geHBydCk7DQo+ICAJc3RydWN0IHhkcl9idWYgKnhkciA9ICZyZXEtPnJxX3NuZF9i dWY7DQo+IEBAIC02MzgsNyArNjM4LDcgQEAgc3RhdGljIGludCB4c191ZHBfc2VuZF9yZXF1ZXN0 KHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4gIAkJLyogU2hvdWxkIHdlIGNhbGwgeHNfY2xvc2Uo KSBoZXJlPyAqLw0KPiAgCQlicmVhazsNCj4gIAljYXNlIC1FQUdBSU46DQo+IC0JCXN0YXR1cyA9 IHhzX25vc3BhY2UodGFzayk7DQo+ICsJCXN0YXR1cyA9IHhzX25vc3BhY2UocmVxLCB0YXNrKTsN Cj4gIAkJYnJlYWs7DQo+ICAJY2FzZSAtRU5FVFVOUkVBQ0g6DQo+ICAJY2FzZSAtRU5PQlVGUzoN Cj4gQEAgLTY1OCw2ICs2NTgsNyBAQCBzdGF0aWMgaW50IHhzX3VkcF9zZW5kX3JlcXVlc3Qoc3Ry dWN0IHJwY190YXNrICp0YXNrKQ0KPiAgDQo+ICAvKioNCj4gICAqIHhzX3RjcF9zZW5kX3JlcXVl c3QgLSB3cml0ZSBhbiBSUEMgcmVxdWVzdCB0byBhIFRDUCBzb2NrZXQNCj4gKyAqIEByZXE6IHBv aW50ZXIgdG8gUlBDIHJlcXVlc3QNCj4gICAqIEB0YXNrOiBhZGRyZXNzIG9mIFJQQyB0YXNrIHRo YXQgbWFuYWdlcyB0aGUgc3RhdGUgb2YgYW4gUlBDIHJlcXVlc3QNCj4gICAqDQo+ICAgKiBSZXR1 cm4gdmFsdWVzOg0KPiBAQCAtNjcwLDkgKzY3MSw4IEBAIHN0YXRpYyBpbnQgeHNfdWRwX3NlbmRf cmVxdWVzdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQo+ICAgKiBYWFg6IEluIHRoZSBjYXNlIG9m IHNvZnQgdGltZW91dHMsIHNob3VsZCB3ZSBldmVudHVhbGx5IGdpdmUgdXANCj4gICAqCWlmIHNl bmRtc2cgaXMgbm90IGFibGUgdG8gbWFrZSBwcm9ncmVzcz8NCj4gICAqLw0KPiAtc3RhdGljIGlu dCB4c190Y3Bfc2VuZF9yZXF1ZXN0KHN0cnVjdCBycGNfdGFzayAqdGFzaykNCj4gK3N0YXRpYyBp bnQgeHNfdGNwX3NlbmRfcmVxdWVzdChzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSwgc3RydWN0IHJwY190 YXNrICp0YXNrKQ0KPiAgew0KPiAtCXN0cnVjdCBycGNfcnFzdCAqcmVxID0gdGFzay0+dGtfcnFz dHA7DQo+ICAJc3RydWN0IHJwY194cHJ0ICp4cHJ0ID0gcmVxLT5ycV94cHJ0Ow0KPiAgCXN0cnVj dCBzb2NrX3hwcnQgKnRyYW5zcG9ydCA9IGNvbnRhaW5lcl9vZih4cHJ0LCBzdHJ1Y3Qgc29ja194 cHJ0LA0KPiB4cHJ0KTsNCj4gIAlzdHJ1Y3QgeGRyX2J1ZiAqeGRyID0gJnJlcS0+cnFfc25kX2J1 ZjsNCj4gQEAgLTY5Nyw3ICs2OTcsNyBAQCBzdGF0aWMgaW50IHhzX3RjcF9zZW5kX3JlcXVlc3Qo c3RydWN0IHJwY190YXNrICp0YXNrKQ0KPiAgCSAqIGNvbXBsZXRlcyB3aGlsZSB0aGUgc29ja2V0 IGhvbGRzIGEgcmVmZXJlbmNlIHRvIHRoZSBwYWdlcywNCj4gIAkgKiB0aGVuIHdlIG1heSBlbmQg dXAgcmVzZW5kaW5nIGNvcnJ1cHRlZCBkYXRhLg0KPiAgCSAqLw0KPiAtCWlmICh0YXNrLT50a19m bGFncyAmIFJQQ19UQVNLX1NFTlQpDQo+ICsJaWYgKHJlcS0+cnFfdGFzay0+dGtfZmxhZ3MgJiBS UENfVEFTS19TRU5UKQ0KDQpDYW4gcmVxIG9yIHJxX3Rhc2sgYmUgbnVsbCBjb21pbmcgaW50byB0 aGlzIGZ1bmN0aW9uPyAgSSdtIHNlZWluZyB0aGUgZm9sbG93aW5nDQpudWxsIHBvaW50ZXIgZGVy ZWZlcmVuY2Ugd2hpbGUgcnVubmluZyBjb25uZWN0YXRob24gdGVzdHMgb24gdjQuMSBhZnRlciB0 aGlzDQpwYXRjaDoNCg0KWyAgIDc3Ljc4MjA0NV0gQlVHOiB1bmFibGUgdG8gaGFuZGxlIGtlcm5l bCBOVUxMIHBvaW50ZXIgZGVyZWZlcmVuY2UgYXQNCjAwMDAwMDAwMDAwMDAwZDQNClsgICA3Ny43 ODIxODRdIFBHRCAwIFA0RCAwIA0KWyAgIDc3Ljc4MjIzMl0gT29wczogMDAwMCBbIzFdIFBSRUVN UFQgU01QIFBUSQ0KWyAgIDc3Ljc4MjI4Nl0gQ1BVOiAwIFBJRDogMTc5MCBDb21tOiBORlN2NCBj YWxsYmFjayBOb3QgdGFpbnRlZCA0LjE5LjAtcmMyLQ0KQU5OQSsgIzEzNTANClsgICA3Ny43ODIz ODJdIEhhcmR3YXJlIG5hbWU6IEJvY2hzIEJvY2hzLCBCSU9TIEJvY2hzIDAxLzAxLzIwMTENClsg ICA3Ny43ODI0NjhdIFJJUDogMDAxMDp4c190Y3Bfc2VuZF9yZXF1ZXN0KzB4NWYvMHgyNTAgW3N1 bnJwY10NClsgICA3Ny43ODI1MzFdIENvZGU6IDc0IDBlIDhiIDk3IDZjIDAxIDAwIDAwIDg1IGQy IDBmIDg0IDIxIDAxIDAwIDAwIDhiIDQzIDQwIDQ4DQo4YiA1MyAwOCA4MyBlOCAwNCAwZCAwMCAw MCAwMCA4MCAwZiBjOCA4OSAwMiA0OCA4YiA4MyA4OCAwMCAwMCAwMCA8MGY+IGI3IGE4IGQ0DQow MCAwMCAwMCA0OSA4YiA4NyA3OCAwNSAwMCAwMCA2NiBjMSBlZCAwYiA4MyBmNSAwMSA4Mw0KWyAg IDc3Ljc4MjcxNl0gUlNQOiAwMDE4OmZmZmZjOTAwMDEwYTNkMTggRUZMQUdTOiAwMDAxMDI4Mg0K WyAgIDc3Ljc4Mjc3NV0gUkFYOiAwMDAwMDAwMDAwMDAwMDAwIFJCWDogZmZmZjg4MDBiYTQ1MmUw MCBSQ1g6IDAwMDAwMDAwMDAwMDAwMDANClsgICA3Ny43ODI4NDNdIFJEWDogZmZmZjg4MDBiODg4 MzAwMCBSU0k6IGZmZmY4ODAwYmE2YjcxMDAgUkRJOiBmZmZmODgwMGJhNDUyZTAwDQpbICAgNzcu NzgyOTExXSBSQlA6IGZmZmY4ODAwYmE0NTJlMDAgUjA4OiAwMDAwMDAxODVlOTc3ZmM5IFIwOTog ZmZmZmZmZmY4MjIyOWM5OA0KWyAgIDc3Ljc4Mjk4MF0gUjEwOiBmZmZmZmZmZjgyMjI5Yzk0IFIx MTogMDAwMDAwMDAwMDAwMDAwMCBSMTI6IGZmZmY4ODAwYmE0NTJlMDgNClsgICA3Ny43ODMwNDhd IFIxMzogZmZmZjg4MDBiYTQ1MmUwMCBSMTQ6IGZmZmZmZmZmYTAzMDM0YjAgUjE1OiBmZmZmODgw MGI0YzExODAwDQpbICAgNzcuNzgzMTE4XSBGUzogIDAwMDAwMDAwMDAwMDAwMDAoMDAwMCkgR1M6 ZmZmZjg4MDBiY2EwMDAwMCgwMDAwKQ0Ka25sR1M6MDAwMDAwMDAwMDAwMDAwMA0KWyAgIDc3Ljc4 MzIxN10gQ1M6ICAwMDEwIERTOiAwMDAwIEVTOiAwMDAwIENSMDogMDAwMDAwMDA4MDA1MDAzMw0K WyAgIDc3Ljc4MzI3OF0gQ1IyOiAwMDAwMDAwMDAwMDAwMGQ0IENSMzogMDAwMDAwMDAwMjAwYTAw MSBDUjQ6IDAwMDAwMDAwMDAxNjA2ZjANClsgICA3Ny43ODMzNTFdIENhbGwgVHJhY2U6DQpbICAg NzcuNzgzNDE4XSAgPyBycGNfd2FrZV91cF9zdGF0dXMrMHg3MC8weDcwIFtzdW5ycGNdDQpbICAg NzcuNzgzNTQyXSAgPyBycGNfY2xudF90ZXN0X2FuZF9hZGRfeHBydCsweGIwLzB4YjAgW3N1bnJw Y10NClsgICA3Ny43ODM2MzRdICB4cHJ0X3RyYW5zbWl0KzB4ODIvMHgyMzAgW3N1bnJwY10NClsg ICA3Ny43ODM3MTFdICA/IHJwY193YWtlX3VwX3N0YXR1cysweDcwLzB4NzAgW3N1bnJwY10NClsg ICA3Ny43ODM3ODZdICA/IHJwY19jbG50X3Rlc3RfYW5kX2FkZF94cHJ0KzB4YjAvMHhiMCBbc3Vu cnBjXQ0KWyAgIDc3Ljc4NDc3Ml0gIGNhbGxfYmNfdHJhbnNtaXQrMHg1MC8weGYwIFtzdW5ycGNd DQpbICAgNzcuNzg1NzM3XSAgX19ycGNfZXhlY3V0ZSsweDdhLzB4MzcwIFtzdW5ycGNdDQpbICAg NzcuNzg2NjI2XSAgPyBrdm1fY2xvY2tfcmVhZCsweDE0LzB4MzANClsgICA3Ny43ODc1NDZdICA/ IGt0aW1lX2dldCsweDMxLzB4OTANClsgICA3Ny43ODgzOTNdICBycGNfcnVuX2JjX3Rhc2srMHg4 NS8weGMwIFtzdW5ycGNdDQpbICAgNzcuNzg5MjYwXSAgYmNfc3ZjX3Byb2Nlc3MrMHgyNzUvMHgy ZjAgW3N1bnJwY10NClsgICA3Ny43OTAyMTZdICBuZnM0MV9jYWxsYmFja19zdmMrMHgxMjAvMHgx OTAgW25mc3Y0XQ0KWyAgIDc3Ljc5MDYzMV0gID8gd2FpdF93b2tlbisweDgwLzB4ODANClsgICA3 Ny43OTA5OThdICA/IG5mc19tYXBfZ2lkX3RvX2dyb3VwKzB4MTEwLzB4MTEwIFtuZnN2NF0NClsg ICA3Ny43OTEzNTZdICBrdGhyZWFkKzB4MTBkLzB4MTMwDQpbICAgNzcuNzkxNzA4XSAgPyBrdGhy ZWFkX3BhcmsrMHg4MC8weDgwDQpbICAgNzcuNzkyMDQ3XSAgcmV0X2Zyb21fZm9yaysweDM1LzB4 NDANClsgICA3Ny43OTIyNzJdIE1vZHVsZXMgbGlua2VkIGluOiBuZnN2MyBycGNzZWNfZ3NzX2ty YjUgbmZzdjQgbmZzIGZzY2FjaGUNCmNmZzgwMjExIHJwY3JkbWEgcmZraWxsIGNyY3QxMGRpZl9w Y2xtdWwgY3JjMzJfcGNsbXVsIGNyYzMyY19pbnRlbA0KZ2hhc2hfY2xtdWxuaV9pbnRlbCBwY2Jj IGpveWRldiBtb3VzZWRldiBhZXNuaV9pbnRlbCBldmRldiBhZXNfeDg2XzY0DQpjcnlwdG9fc2lt ZCBpbnB1dF9sZWRzIGNyeXB0ZCBsZWRfY2xhc3MgbWFjX2hpZCBnbHVlX2hlbHBlciBwc21vdXNl IHBjc3Brcg0KaW50ZWxfYWdwIG5mc2QgaW50ZWxfZ3R0IGkyY19waWl4NCBidXR0b24gYXV0aF9y cGNnc3MgbmZzX2FjbCBsb2NrZCBncmFjZSBzdW5ycGMNCnNjaF9mcV9jb2RlbCBpcF90YWJsZXMg eF90YWJsZXMgYXRhX2dlbmVyaWMgcGF0YV9hY3BpIHNlcmlvX3JhdyBhdGtiZCBsaWJwczINCmF0 YV9waWl4IGxpYmF0YSBpODA0MiBmbG9wcHkgc2VyaW8gdWhjaV9oY2Qgc2NzaV9tb2QgZWhjaV9w Y2kgZWhjaV9oY2QgdXNiY29yZQ0KdXNiX2NvbW1vbiB4ZnMgdmlydGlvX2JhbGxvb24gdmlydGlv X25ldCBuZXRfZmFpbG92ZXIgZmFpbG92ZXIgdmlydGlvX3BjaQ0KdmlydGlvX2JsayB2aXJ0aW9f cmluZyB2aXJ0aW8NClsgICA3Ny43OTQzOTddIENSMjogMDAwMDAwMDAwMDAwMDBkNA0KWyAgIDc3 Ljc5NDY1MF0gLS0tWyBlbmQgdHJhY2UgOGQ4ODg4YjZiZGU4NGQzMyBdLS0tDQoNCg0KVGhhbmtz LA0KQW5uYQ0KPiAgCQl6ZXJvY29weSA9IGZhbHNlOw0KPiAgDQo+ICAJaWYgKHRlc3RfYml0KFhQ UlRfU09DS19VUERfVElNRU9VVCwgJnRyYW5zcG9ydC0+c29ja19zdGF0ZSkpDQo+IEBAIC03NjEs NyArNzYxLDcgQEAgc3RhdGljIGludCB4c190Y3Bfc2VuZF9yZXF1ZXN0KHN0cnVjdCBycGNfdGFz ayAqdGFzaykNCj4gIAkJLyogU2hvdWxkIHdlIGNhbGwgeHNfY2xvc2UoKSBoZXJlPyAqLw0KPiAg CQlicmVhazsNCj4gIAljYXNlIC1FQUdBSU46DQo+IC0JCXN0YXR1cyA9IHhzX25vc3BhY2UodGFz ayk7DQo+ICsJCXN0YXR1cyA9IHhzX25vc3BhY2UocmVxLCB0YXNrKTsNCj4gIAkJYnJlYWs7DQo+ ICAJY2FzZSAtRUNPTk5SRVNFVDoNCj4gIAljYXNlIC1FQ09OTlJFRlVTRUQ6DQo+IEBAIC0yNzA2 LDkgKzI3MDYsOCBAQCBzdGF0aWMgaW50IGJjX3NlbmR0byhzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSkN Cj4gIC8qDQo+ICAgKiBUaGUgc2VuZCByb3V0aW5lLiBCb3Jyb3dzIGZyb20gc3ZjX3NlbmQNCj4g ICAqLw0KPiAtc3RhdGljIGludCBiY19zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY190YXNrICp0YXNr KQ0KPiArc3RhdGljIGludCBiY19zZW5kX3JlcXVlc3Qoc3RydWN0IHJwY19ycXN0ICpyZXEsIHN0 cnVjdCBycGNfdGFzayAqdGFzaykNCj4gIHsNCj4gLQlzdHJ1Y3QgcnBjX3Jxc3QgKnJlcSA9IHRh c2stPnRrX3Jxc3RwOw0KPiAgCXN0cnVjdCBzdmNfeHBydAkqeHBydDsNCj4gIAlpbnQgbGVuOw0K PiAgDQo= ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() 2018-09-06 18:49 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Schumaker, Anna @ 2018-09-06 18:57 ` Trond Myklebust 0 siblings, 0 replies; 45+ messages in thread From: Trond Myklebust @ 2018-09-06 18:57 UTC (permalink / raw) To: Schumaker, Anna, linux-nfs On Thu, 2018-09-06 at 18:49 +0000, Schumaker, Anna wrote: > Hi Trond, > > On Tue, 2018-09-04 at 17:05 -0400, Trond Myklebust wrote: > > When we shift to using the transmit queue, then the task that holds > > the > > write lock will not necessarily be the same as the one being > > transmitted. > > > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> > > --- > > include/linux/sunrpc/xprt.h | 2 +- > > net/sunrpc/xprt.c | 2 +- > > net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 3 +-- > > net/sunrpc/xprtrdma/transport.c | 5 ++-- > > net/sunrpc/xprtsock.c | 27 +++++++++++------- > > ---- > > 5 files changed, 18 insertions(+), 21 deletions(-) > > > > diff --git a/include/linux/sunrpc/xprt.h > > b/include/linux/sunrpc/xprt.h > > index 81a6c2c8dfc7..6d91acfe0644 100644 > > --- a/include/linux/sunrpc/xprt.h > > +++ b/include/linux/sunrpc/xprt.h > > @@ -140,7 +140,7 @@ struct rpc_xprt_ops { > > void (*connect)(struct rpc_xprt *xprt, struct > > rpc_task > > *task); > > int (*buf_alloc)(struct rpc_task *task); > > void (*buf_free)(struct rpc_task *task); > > - int (*send_request)(struct rpc_task *task); > > + int (*send_request)(struct rpc_rqst *req, struct > > rpc_task > > *task); > > void (*set_retrans_timeout)(struct rpc_task > > *task); > > void (*timer)(struct rpc_xprt *xprt, struct > > rpc_task > > *task); > > void (*release_request)(struct rpc_task *task); > > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c > > index 8e8c345eedf7..7c80f93562e5 100644 > > --- a/net/sunrpc/xprt.c > > +++ b/net/sunrpc/xprt.c > > @@ -1170,7 +1170,7 @@ void xprt_transmit(struct rpc_task *task) > > } > > > > connect_cookie = xprt->connect_cookie; > > - status = xprt->ops->send_request(task); > > + status = xprt->ops->send_request(req, task); > > trace_xprt_transmit(xprt, req->rq_xid, status); > > if (status != 0) { > > task->tk_status = status; > > diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c > > b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c > > index 09b12b7568fe..d1618c70edb4 100644 > > --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c > > +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c > > @@ -215,9 +215,8 @@ rpcrdma_bc_send_request(struct svcxprt_rdma > > *rdma, struct > > rpc_rqst *rqst) > > * connection. > > */ > > static int > > -xprt_rdma_bc_send_request(struct rpc_task *task) > > +xprt_rdma_bc_send_request(struct rpc_rqst *rqst, struct rpc_task > > *task) > > { > > - struct rpc_rqst *rqst = task->tk_rqstp; > > struct svc_xprt *sxprt = rqst->rq_xprt->bc_xprt; > > struct svcxprt_rdma *rdma; > > int ret; > > diff --git a/net/sunrpc/xprtrdma/transport.c > > b/net/sunrpc/xprtrdma/transport.c > > index 143ce2579ba9..fa684bf4d090 100644 > > --- a/net/sunrpc/xprtrdma/transport.c > > +++ b/net/sunrpc/xprtrdma/transport.c > > @@ -706,9 +706,8 @@ xprt_rdma_free(struct rpc_task *task) > > * sent. Do not try to send this message again. > > */ > > static int > > -xprt_rdma_send_request(struct rpc_task *task) > > +xprt_rdma_send_request(struct rpc_rqst *rqst, struct rpc_task > > *task) > > { > > - struct rpc_rqst *rqst = task->tk_rqstp; > > struct rpc_xprt *xprt = rqst->rq_xprt; > > struct rpcrdma_req *req = rpcr_to_rdmar(rqst); > > struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt); > > @@ -741,7 +740,7 @@ xprt_rdma_send_request(struct rpc_task *task) > > /* An RPC with no reply will throw off credit accounting, > > * so drop the connection to reset the credit grant. > > */ > > - if (!rpc_reply_expected(task)) > > + if (!rpc_reply_expected(rqst->rq_task)) > > goto drop_connection; > > return 0; > > > > diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c > > index 8d6404259ff9..b8143eded4af 100644 > > --- a/net/sunrpc/xprtsock.c > > +++ b/net/sunrpc/xprtsock.c > > @@ -449,12 +449,12 @@ static void xs_nospace_callback(struct > > rpc_task *task) > > > > /** > > * xs_nospace - place task on wait queue if transmit was > > incomplete > > + * @req: pointer to RPC request > > * @task: task to put to sleep > > * > > */ > > -static int xs_nospace(struct rpc_task *task) > > +static int xs_nospace(struct rpc_rqst *req, struct rpc_task *task) > > { > > - struct rpc_rqst *req = task->tk_rqstp; > > struct rpc_xprt *xprt = req->rq_xprt; > > struct sock_xprt *transport = container_of(xprt, struct > > sock_xprt, > > xprt); > > struct sock *sk = transport->inet; > > @@ -513,6 +513,7 @@ static inline void > > xs_encode_stream_record_marker(struct > > xdr_buf *buf) > > > > /** > > * xs_local_send_request - write an RPC request to an AF_LOCAL > > socket > > + * @req: pointer to RPC request > > * @task: RPC task that manages the state of an RPC request > > * > > * Return values: > > @@ -522,9 +523,8 @@ static inline void > > xs_encode_stream_record_marker(struct > > xdr_buf *buf) > > * ENOTCONN: Caller needs to invoke connect logic then call > > again > > * other: Some other error occured, the request was not > > sent > > */ > > -static int xs_local_send_request(struct rpc_task *task) > > +static int xs_local_send_request(struct rpc_rqst *req, struct > > rpc_task *task) > > { > > - struct rpc_rqst *req = task->tk_rqstp; > > struct rpc_xprt *xprt = req->rq_xprt; > > struct sock_xprt *transport = > > container_of(xprt, struct sock_xprt, > > xprt); > > @@ -569,7 +569,7 @@ static int xs_local_send_request(struct > > rpc_task *task) > > case -ENOBUFS: > > break; > > case -EAGAIN: > > - status = xs_nospace(task); > > + status = xs_nospace(req, task); > > break; > > default: > > dprintk("RPC: sendmsg returned unrecognized error > > %d\n", > > @@ -585,6 +585,7 @@ static int xs_local_send_request(struct > > rpc_task *task) > > > > /** > > * xs_udp_send_request - write an RPC request to a UDP socket > > + * @req: pointer to RPC request > > * @task: address of RPC task that manages the state of an RPC > > request > > * > > * Return values: > > @@ -594,9 +595,8 @@ static int xs_local_send_request(struct > > rpc_task *task) > > * ENOTCONN: Caller needs to invoke connect logic then call > > again > > * other: Some other error occurred, the request was not > > sent > > */ > > -static int xs_udp_send_request(struct rpc_task *task) > > +static int xs_udp_send_request(struct rpc_rqst *req, struct > > rpc_task *task) > > { > > - struct rpc_rqst *req = task->tk_rqstp; > > struct rpc_xprt *xprt = req->rq_xprt; > > struct sock_xprt *transport = container_of(xprt, struct > > sock_xprt, > > xprt); > > struct xdr_buf *xdr = &req->rq_snd_buf; > > @@ -638,7 +638,7 @@ static int xs_udp_send_request(struct rpc_task > > *task) > > /* Should we call xs_close() here? */ > > break; > > case -EAGAIN: > > - status = xs_nospace(task); > > + status = xs_nospace(req, task); > > break; > > case -ENETUNREACH: > > case -ENOBUFS: > > @@ -658,6 +658,7 @@ static int xs_udp_send_request(struct rpc_task > > *task) > > > > /** > > * xs_tcp_send_request - write an RPC request to a TCP socket > > + * @req: pointer to RPC request > > * @task: address of RPC task that manages the state of an RPC > > request > > * > > * Return values: > > @@ -670,9 +671,8 @@ static int xs_udp_send_request(struct rpc_task > > *task) > > * XXX: In the case of soft timeouts, should we eventually give up > > * if sendmsg is not able to make progress? > > */ > > -static int xs_tcp_send_request(struct rpc_task *task) > > +static int xs_tcp_send_request(struct rpc_rqst *req, struct > > rpc_task *task) > > { > > - struct rpc_rqst *req = task->tk_rqstp; > > struct rpc_xprt *xprt = req->rq_xprt; > > struct sock_xprt *transport = container_of(xprt, struct > > sock_xprt, > > xprt); > > struct xdr_buf *xdr = &req->rq_snd_buf; > > @@ -697,7 +697,7 @@ static int xs_tcp_send_request(struct rpc_task > > *task) > > * completes while the socket holds a reference to the pages, > > * then we may end up resending corrupted data. > > */ > > - if (task->tk_flags & RPC_TASK_SENT) > > + if (req->rq_task->tk_flags & RPC_TASK_SENT) > > Can req or rq_task be null coming into this function? I'm seeing the > following > null pointer dereference while running connectathon tests on v4.1 > after this > patch: The backchannels do need PATCH 24/34 ^ permalink raw reply [flat|nested] 45+ messages in thread
end of thread, other threads:[~2018-09-06 23:34 UTC | newest] Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-09-04 21:05 [PATCH v2 00/34] Convert RPC client transmission to a queued model Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 01/34] SUNRPC: Clean up initialisation of the struct rpc_rqst Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 02/34] SUNRPC: If there is no reply expected, bail early from call_decode Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 03/34] SUNRPC: The transmitted message must lie in the RPCSEC window of validity Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 04/34] SUNRPC: Simplify identification of when the message send/receive is complete Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 05/34] SUNRPC: Avoid holding locks across the XDR encoding of the RPC message Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 06/34] SUNRPC: Rename TCP receive-specific state variables Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 07/34] SUNRPC: Move reset of TCP state variables into the reconnect code Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 08/34] SUNRPC: Add socket transmit queue offset tracking Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 09/34] SUNRPC: Simplify dealing with aborted partially transmitted messages Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 10/34] SUNRPC: Refactor the transport request pinning Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 11/34] SUNRPC: Add a helper to wake up a sleeping rpc_task and set its status Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 12/34] SUNRPC: Don't wake queued RPC calls multiple times in xprt_transmit Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 13/34] SUNRPC: Rename xprt->recv_lock to xprt->queue_lock Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 14/34] SUNRPC: Refactor xprt_transmit() to remove the reply queue code Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 15/34] SUNRPC: Refactor xprt_transmit() to remove wait for reply code Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 16/34] SUNRPC: Minor cleanup for call_transmit() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 17/34] SUNRPC: Distinguish between the slot allocation list and receive queue Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 18/34] NFS: Add a transmission queue for RPC requests Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 19/34] SUNRPC: Refactor RPC call encoding Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 21/34] SUNRPC: Don't reset the request 'bytes_sent' counter when releasing XPRT_LOCK Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 22/34] SUNRPC: Simplify xprt_prepare_transmit() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 24/34] SUNRPC: Fix up the back channel transmit Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 25/34] SUNRPC: Support for congestion control when queuing is enabled Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 26/34] SUNRPC: Improve latency for interactive tasks Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 27/34] SUNRPC: Allow calls to xprt_transmit() to drain the entire transmit queue Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 28/34] SUNRPC: Queue the request for transmission immediately after encoding Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 30/34] SUNRPC: Allow soft RPC calls to time out when waiting for the XPRT_LOCK Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 31/34] SUNRPC: Turn off throttling of RPC slots for TCP sockets Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 32/34] SUNRPC: Clean up transport write space handling Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 33/34] SUNRPC: Cleanup: remove the unused 'task' argument from the request_send() Trond Myklebust 2018-09-04 21:05 ` [PATCH v2 34/34] SUNRPC: Queue fairness for all Trond Myklebust 2018-09-06 14:17 ` [PATCH v2 29/34] SUNRPC: Convert the xprt->sending queue back to an ordinary wait queue Schumaker, Anna 2018-09-06 14:23 ` Schumaker, Anna 2018-09-06 14:23 ` Schumaker, Anna 2018-09-05 14:30 ` [PATCH v2 23/34] SUNRPC: Move RPC retransmission stat counter to xprt_transmit() Chuck Lever 2018-09-05 15:28 ` Trond Myklebust 2018-09-05 15:31 ` Chuck Lever 2018-09-05 16:07 ` Trond Myklebust 2018-09-05 16:34 ` Chuck Lever 2018-09-06 18:49 ` [PATCH v2 20/34] SUNRPC: Treat the task and request as separate in the xprt_ops->send_request() Schumaker, Anna 2018-09-06 18:57 ` Trond Myklebust
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.