[RFC v3 1/2] NFS: Allow multiple connections to NFSv3 and NFSv4.0 servers

[RFC v3 1/2] NFS: Allow multiple connections to NFSv3 and NFSv4.0 servers

[Bennett Amodio]

Enable nconnect mount option and multipathing behavior for NFSv3 and NFSv4.

Signed-off-by: Jui-Yu Chang <juchang@purestorage.com>
---
 fs/nfs/client.c     | 3 +++
 fs/nfs/nfs4client.c | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 08baa66..8d11a11 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -653,6 +653,9 @@ static int nfs_init_server(struct nfs_server *server,
  struct nfs_client *clp;
  int error;

+ if (data->nfs_server.protocol =3D=3D XPRT_TRANSPORT_TCP)
+ cl_init.nconnect =3D data->nfs_server.nconnect;
+
  nfs_init_timeout_values(&timeparms, data->nfs_server.protocol,
  data->timeo, data->retrans);
  if (data->flags & NFS_MOUNT_NORESVPORT)
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index f11dec1..6ef1baa 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -849,7 +849,7 @@ static int nfs4_set_client(struct nfs_server *server,
  };
  struct nfs_client *clp;

- if (minorversion > 0 && proto =3D=3D XPRT_TRANSPORT_TCP)
+ if (proto =3D=3D XPRT_TRANSPORT_TCP)
  cl_init.nconnect =3D nconnect;
  if (server->flags & NFS_MOUNT_NORESVPORT)
  set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags);
--=20
1.9.1

On Tue, Aug 15, 2017 at 5:46 PM, Bennett Amodio <bamodio@purestorage.com> w=
rote:
> After seeing Trond=E2=80=99s patches for NFS multipathing on NFSv4.1, we
> decided to try using the same concept for NFSv3/4.  The primary issue
> we identified was XID collision in the duplicate request cache (replay
> cache) for NFSv3/4.  In NFSv3/4, entries are hashed based on XID
> instead of the slot ID and sequence ID that NFSv4.1 uses.  Since the
> XIDs are generated by the RPC transports, and Trond=E2=80=99s patches cre=
ate
> multiple transports for multipathing, different transports can end up
> using an overlapping set of XIDs.
>
>
> To fix this, we apply a mask to XIDs. Each transport is constrained to
> its own segment of the total XID range, and they can never overlap.
> In terms of loss of entropy, by masking out just enough bits from the
> XID, we are convinced that the probability of XID wraparound or
> collision on NFS client restart has not increased to a problematic
> level (so long as the RPCs are distributed round-robin, as in Trond=E2=80=
=99s
> patches).
>
>
> We tested multipathing out and discovered that it enables NFS to get
> more bandwidth on a bonded interface (instead of using only one
> physical link, it can use multiple).  Specifically, we tested on a
> setup where the client was connected to the server via 4 bonded 10Gb/s
> links.  Without multipathing, the client could only achieve 10Gb/s
> (using one physical link).  With multipathing, the client was able to
> achieve a maximum of close to 40Gb/s.
>
>
> However, although the maximum performance was close to 40Gb/s,
> achieving an average throughput of even 30Gb/s required many
> connections.  The performance of individual trials had a high
> variance.  We traced this uneven performance to colliding network
> paths.  With round-robin distribution of RPCs, no single TCP
> connection can exceed the performance of the slowest one.  If the
> connections are distributed unevenly across network paths, some
> connections can bottleneck others.  To solve this problem, we are
> currently working on patches to provide load-balancing as an
> alternative to round-robin for distributing RPCs.
>
>
> To use these patches, you first have to apply Trond's 5 patches
> (Available at https://www.spinics.net/lists/linux-nfs/msg63368.html).
> Let us know what you think or if you have any ideas for improving
> this.
>
>
> Jui-Yu Chang (1):
>   NFS: Allow multiple connections to NFSv3 and NFSv4.0 servers
>
> Bennett Amodio (1):
>   SUNRPC: Mask XIDs to prevent replay cache collision
>
>  fs/nfs/client.c             |  3 +++
>  fs/nfs/nfs4client.c         |  2 +-
>  include/linux/sunrpc/xprt.h |  5 +++++
>  net/sunrpc/clnt.c           |  8 ++++++++
>  net/sunrpc/xprt.c           | 14 ++++++--------
>  5 files changed, 23 insertions(+), 9 deletions(-)
>
> --
> 1.9.1