[PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Benny Halevy]

Hi Bruce,

Here's the latest server patches implementing the NFSv4.1
Sessions features.

This patchset is based over your for-2.6.30 branch
and is also available from
git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
(tagged as nfsd41-for-2.6.30-2009-03-27)

The implementation status highlights are captured in
Documentation/filesystems/nfs41-server.txt.
The gist of it is that the implementation is functional
to the level of passing Connectathon tests with v2,v3,v4,v4.1
(tested with CONFIG_NFS_V4_1 defined to 'y')

The patches are listed below.

Thanks for your review and help with this project so far.
Kudos to all the contributors (Authors/SOBs: Andy Adamson,
Dean Hildebrand, Marc Eshel, Mike Sager, Ricardo Labiaga, and
myself)  This is a pretty significant milestone for us
(well, the real one will be when all this ends up working
flawlessly in the kernel. :)

Benny

[PATCH 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY
[PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt
[PATCH 03/47] nfsd: embed nfsd4_current_state in nfsd4_compoundres
[PATCH 04/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state
[PATCH 05/47] nfs41: common protocol definitions
[PATCH 06/47] nfsd41: change NFSERR_REPLAY_ME
[PATCH 07/47] nfsd41: Add Kconfig symbols for NFSv4.1
[PATCH 08/47] nfsd41: define nfs41 error codes
[PATCH 09/47] nfsd41: sessions basic data types
[PATCH 10/47] nfsd41: introduce nfs4_client cl_sessions list
[PATCH 11/47] nfsd41: release_session when client is expired
[PATCH 12/47] nfsd41: sessionid hashing
[PATCH 13/47] nfsd41: xdr infrastructure
[PATCH 14/47] nfsd: remove nfsd4_ops array size
[PATCH 15/47] nfsd41: proc stubs
[PATCH 16/47] nfsd41: exchange_id operation
[PATCH 17/47] nfsd41: match clientid establishment method
[PATCH 18/47] nfsd41: sequence operation
[PATCH 19/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules
[PATCH 20/47] nfsd41: DRC save, restore, and clear functions
[PATCH 21/47] nfsd41: hard page limit for DRC
[PATCH 22/47] nfsd41: nfsd DRC logic
[PATCH 23/47] nfsd41: clear DRC cache on free_session
[PATCH 24/47] nfsd41: create_session operation
[PATCH 25/47] nfsd41: Add a create session replay cache
[PATCH 26/47] nfsd41: non-page DRC for solo sequence responses
[PATCH 27/47] nfsd41: destroy_session operation
[PATCH 28/47] nfsd41: stateid handling
[PATCH 29/47] nfsd41: check encode size for sessions maxresponse cached
[PATCH 30/47] nfsd41: clientid handling
[PATCH 31/47] nfsd41: access_valid
[PATCH 32/47] nfsd41: add OPEN4_SHARE_ACCESS_WANT nfs4_stateid bmap
[PATCH 33/47] nfsd41: provide support for minor version 1 at rpc level
[PATCH 34/47] nfsd: cleanup nfs4.0 callback encode routines
[PATCH 35/47] nfsd: minorversion support for the back channel
[PATCH 36/47] nfsd41: sunrpc: Added rpc server-side backchannel handling
[PATCH 37/47] nfsd41: callback infrastructure
[PATCH 38/47] nfsd41: Remember the auth flavor to use for callbacks
[PATCH 39/47] nfsd41: introduce cl_cb_mutex
[PATCH 40/47] nfsd41: cb_sequence callback
[PATCH 41/47] nfsd41: introduce nfs4_cb_call_sync for nfs4 and nfs41
[PATCH 42/47] nfsd41: cb_recall callback
[PATCH 43/47] nfsd41: pass writable attrs mask to nfsd4_decode_fattr
[PATCH 44/47] nfsd41: support for 3-word long attribute bitmask
[PATCH 45/47] nfsd41: SUPPATTR_EXCLCREAT attribute
[PATCH 46/47] nfsd41: CREATE_EXCLUSIVE4_1
[PATCH 47/47] nfsd41: Documentation/filesystems/nfs41-server.txt

[PATCH 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

On an NFSv4.1 server cache miss that causes an upcall, NFS4ERR_DELAY will be
returned. It is up to the NFSv4.1 client to resend only the operations that
have not been processed.

Initialize rq_usedeferral to 1 in svc_process(). rq_usedeferral was initialized
to 0, which means that the first NFSv4.0 or NFSv4.1 rpc would be dropped due
to the logic in svc_defer(). Will be turned off in nfsd4_proc_compound() only
when NFSv4 Sessions are used.

Note: this isn't an adequate solution on its own. It's acceptable as a way
to get some minimal 4.1 up and working, but we're going to have to find a
way to avoid returning DELAY in all common cases before 4.1 can really be
considered ready.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: reverse rq_nodeferral negative logic]
    use positive rq_usedeferral logic instead.
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[sunrpc: initialize rq_usedeferral]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c         |    8 ++++++++
 include/linux/sunrpc/svc.h |    1 +
 net/sunrpc/svc.c           |    2 ++
 net/sunrpc/svc_xprt.c      |    2 +-
 4 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f156b85..7839654 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -873,6 +873,8 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	resp->tag = args->tag;
 	resp->opcnt = 0;
 	resp->rqstp = rqstp;
+	/* Use the deferral mechanism only for NFSv4.0 compounds */
+	rqstp->rq_usedeferral = (args->minorversion == 0);
 
 	/*
 	 * According to RFC3010, this takes precedence over all other errors.
@@ -957,10 +959,16 @@ encode_op:
 
 		nfsd4_increment_op_stats(op->opnum);
 	}
+	if (!rqstp->rq_usedeferral && status == nfserr_dropit) {
+		dprintk("%s Dropit - send NFS4ERR_DELAY\n", __func__);
+		status = nfserr_jukebox;
+	}
 
 	cstate_free(cstate);
 out:
 	nfsd4_release_compoundargs(args);
+	/* Reset deferral mechanism for RPC deferrals */
+	rqstp->rq_usedeferral = 1;
 	dprintk("nfsv4 compound returned %d\n", ntohl(status));
 	return status;
 }
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 9f9f699..815dd58 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -230,6 +230,7 @@ struct svc_rqst {
 	struct svc_cred		rq_cred;	/* auth info */
 	void *			rq_xprt_ctxt;	/* transport specific context ptr */
 	struct svc_deferred_req*rq_deferred;	/* deferred request we are replaying */
+	int			rq_usedeferral;	/* use deferral */
 
 	size_t			rq_xprt_hlen;	/* xprt header len */
 	struct xdr_buf		rq_arg;
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index c51fed4..6334858 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -1023,6 +1023,8 @@ svc_process(struct svc_rqst *rqstp)
 	rqstp->rq_res.tail[0].iov_len = 0;
 	/* Will be turned off only in gss privacy case: */
 	rqstp->rq_splice_ok = 1;
+	/* Will be turned off only when NFSv4 Sessions are used */
+	rqstp->rq_usedeferral = 1;
 
 	/* Setup reply header */
 	rqstp->rq_xprt->xpt_ops->xpo_prep_reply_hdr(rqstp);
diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
index 1e66f24..600d091 100644
--- a/net/sunrpc/svc_xprt.c
+++ b/net/sunrpc/svc_xprt.c
@@ -974,7 +974,7 @@ static struct cache_deferred_req *svc_defer(struct cache_req *req)
 	struct svc_rqst *rqstp = container_of(req, struct svc_rqst, rq_chandle);
 	struct svc_deferred_req *dr;
 
-	if (rqstp->rq_arg.page_len)
+	if (rqstp->rq_arg.page_len || !rqstp->rq_usedeferral)
 		return NULL; /* if more than a page, give up FIXME */
 	if (rqstp->rq_deferred) {
 		dr = rqstp->rq_deferred;
-- 
1.6.2.1

Re: [PATCH 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY

[J. Bruce Fields]

On Fri, Mar 27, 2009 at 06:01:37AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> On an NFSv4.1 server cache miss that causes an upcall, NFS4ERR_DELAY will be
> returned. It is up to the NFSv4.1 client to resend only the operations that
> have not been processed.
> 
> Initialize rq_usedeferral to 1 in svc_process(). rq_usedeferral was initialized
> to 0, which means that the first NFSv4.0 or NFSv4.1 rpc would be dropped due
> to the logic in svc_defer().

Nit: is this a comment about a change from a previous version of the
patch?  It doesn't make much sense to a reader who hasn't seen that
previous version.

> Will be turned off in nfsd4_proc_compound() only
> when NFSv4 Sessions are used.
> 
> Note: this isn't an adequate solution on its own. It's acceptable as a way
> to get some minimal 4.1 up and working, but we're going to have to find a
> way to avoid returning DELAY in all common cases before 4.1 can really be
> considered ready.
> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: reverse rq_nodeferral negative logic]
>     use positive rq_usedeferral logic instead.
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [sunrpc: initialize rq_usedeferral]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>

I'd be inclined to just collapse that into the two signed-off-by's, and
drop the patch-history information.

--b.

> ---
>  fs/nfsd/nfs4proc.c         |    8 ++++++++
>  include/linux/sunrpc/svc.h |    1 +
>  net/sunrpc/svc.c           |    2 ++
>  net/sunrpc/svc_xprt.c      |    2 +-
>  4 files changed, 12 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index f156b85..7839654 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -873,6 +873,8 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  	resp->tag = args->tag;
>  	resp->opcnt = 0;
>  	resp->rqstp = rqstp;
> +	/* Use the deferral mechanism only for NFSv4.0 compounds */
> +	rqstp->rq_usedeferral = (args->minorversion == 0);
>  
>  	/*
>  	 * According to RFC3010, this takes precedence over all other errors.
> @@ -957,10 +959,16 @@ encode_op:
>  
>  		nfsd4_increment_op_stats(op->opnum);
>  	}
> +	if (!rqstp->rq_usedeferral && status == nfserr_dropit) {
> +		dprintk("%s Dropit - send NFS4ERR_DELAY\n", __func__);
> +		status = nfserr_jukebox;
> +	}
>  
>  	cstate_free(cstate);
>  out:
>  	nfsd4_release_compoundargs(args);
> +	/* Reset deferral mechanism for RPC deferrals */
> +	rqstp->rq_usedeferral = 1;
>  	dprintk("nfsv4 compound returned %d\n", ntohl(status));
>  	return status;
>  }
> diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
> index 9f9f699..815dd58 100644
> --- a/include/linux/sunrpc/svc.h
> +++ b/include/linux/sunrpc/svc.h
> @@ -230,6 +230,7 @@ struct svc_rqst {
>  	struct svc_cred		rq_cred;	/* auth info */
>  	void *			rq_xprt_ctxt;	/* transport specific context ptr */
>  	struct svc_deferred_req*rq_deferred;	/* deferred request we are replaying */
> +	int			rq_usedeferral;	/* use deferral */
>  
>  	size_t			rq_xprt_hlen;	/* xprt header len */
>  	struct xdr_buf		rq_arg;
> diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
> index c51fed4..6334858 100644
> --- a/net/sunrpc/svc.c
> +++ b/net/sunrpc/svc.c
> @@ -1023,6 +1023,8 @@ svc_process(struct svc_rqst *rqstp)
>  	rqstp->rq_res.tail[0].iov_len = 0;
>  	/* Will be turned off only in gss privacy case: */
>  	rqstp->rq_splice_ok = 1;
> +	/* Will be turned off only when NFSv4 Sessions are used */
> +	rqstp->rq_usedeferral = 1;
>  
>  	/* Setup reply header */
>  	rqstp->rq_xprt->xpt_ops->xpo_prep_reply_hdr(rqstp);
> diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
> index 1e66f24..600d091 100644
> --- a/net/sunrpc/svc_xprt.c
> +++ b/net/sunrpc/svc_xprt.c
> @@ -974,7 +974,7 @@ static struct cache_deferred_req *svc_defer(struct cache_req *req)
>  	struct svc_rqst *rqstp = container_of(req, struct svc_rqst, rq_chandle);
>  	struct svc_deferred_req *dr;
>  
> -	if (rqstp->rq_arg.page_len)
> +	if (rqstp->rq_arg.page_len || !rqstp->rq_usedeferral)
>  		return NULL; /* if more than a page, give up FIXME */
>  	if (rqstp->rq_deferred) {
>  		dr = rqstp->rq_deferred;
> -- 
> 1.6.2.1
> 

Re: [PATCH 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY

[Benny Halevy]

On Mar. 28, 2009, 3:04 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Fri, Mar 27, 2009 at 06:01:37AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> On an NFSv4.1 server cache miss that causes an upcall, NFS4ERR_DELAY will be
>> returned. It is up to the NFSv4.1 client to resend only the operations that
>> have not been processed.
>>
>> Initialize rq_usedeferral to 1 in svc_process(). rq_usedeferral was initialized
>> to 0, which means that the first NFSv4.0 or NFSv4.1 rpc would be dropped due
>> to the logic in svc_defer().
> 
> Nit: is this a comment about a change from a previous version of the
> patch?  It doesn't make much sense to a reader who hasn't seen that
> previous version.

Yes.  I guess we can drop the second sentence in the paragraph above
from the final version.

Benny

> 
>> Will be turned off in nfsd4_proc_compound() only
>> when NFSv4 Sessions are used.
>>
>> Note: this isn't an adequate solution on its own. It's acceptable as a way
>> to get some minimal 4.1 up and working, but we're going to have to find a
>> way to avoid returning DELAY in all common cases before 4.1 can really be
>> considered ready.
>>
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfsd41: reverse rq_nodeferral negative logic]
>>     use positive rq_usedeferral logic instead.
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [sunrpc: initialize rq_usedeferral]
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> 
> I'd be inclined to just collapse that into the two signed-off-by's, and
> drop the patch-history information.
> 
> --b.
> 
>> ---
>>  fs/nfsd/nfs4proc.c         |    8 ++++++++
>>  include/linux/sunrpc/svc.h |    1 +
>>  net/sunrpc/svc.c           |    2 ++
>>  net/sunrpc/svc_xprt.c      |    2 +-
>>  4 files changed, 12 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>> index f156b85..7839654 100644
>> --- a/fs/nfsd/nfs4proc.c
>> +++ b/fs/nfsd/nfs4proc.c
>> @@ -873,6 +873,8 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>>  	resp->tag = args->tag;
>>  	resp->opcnt = 0;
>>  	resp->rqstp = rqstp;
>> +	/* Use the deferral mechanism only for NFSv4.0 compounds */
>> +	rqstp->rq_usedeferral = (args->minorversion == 0);
>>  
>>  	/*
>>  	 * According to RFC3010, this takes precedence over all other errors.
>> @@ -957,10 +959,16 @@ encode_op:
>>  
>>  		nfsd4_increment_op_stats(op->opnum);
>>  	}
>> +	if (!rqstp->rq_usedeferral && status == nfserr_dropit) {
>> +		dprintk("%s Dropit - send NFS4ERR_DELAY\n", __func__);
>> +		status = nfserr_jukebox;
>> +	}
>>  
>>  	cstate_free(cstate);
>>  out:
>>  	nfsd4_release_compoundargs(args);
>> +	/* Reset deferral mechanism for RPC deferrals */
>> +	rqstp->rq_usedeferral = 1;
>>  	dprintk("nfsv4 compound returned %d\n", ntohl(status));
>>  	return status;
>>  }
>> diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
>> index 9f9f699..815dd58 100644
>> --- a/include/linux/sunrpc/svc.h
>> +++ b/include/linux/sunrpc/svc.h
>> @@ -230,6 +230,7 @@ struct svc_rqst {
>>  	struct svc_cred		rq_cred;	/* auth info */
>>  	void *			rq_xprt_ctxt;	/* transport specific context ptr */
>>  	struct svc_deferred_req*rq_deferred;	/* deferred request we are replaying */
>> +	int			rq_usedeferral;	/* use deferral */
>>  
>>  	size_t			rq_xprt_hlen;	/* xprt header len */
>>  	struct xdr_buf		rq_arg;
>> diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
>> index c51fed4..6334858 100644
>> --- a/net/sunrpc/svc.c
>> +++ b/net/sunrpc/svc.c
>> @@ -1023,6 +1023,8 @@ svc_process(struct svc_rqst *rqstp)
>>  	rqstp->rq_res.tail[0].iov_len = 0;
>>  	/* Will be turned off only in gss privacy case: */
>>  	rqstp->rq_splice_ok = 1;
>> +	/* Will be turned off only when NFSv4 Sessions are used */
>> +	rqstp->rq_usedeferral = 1;
>>  
>>  	/* Setup reply header */
>>  	rqstp->rq_xprt->xpt_ops->xpo_prep_reply_hdr(rqstp);
>> diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
>> index 1e66f24..600d091 100644
>> --- a/net/sunrpc/svc_xprt.c
>> +++ b/net/sunrpc/svc_xprt.c
>> @@ -974,7 +974,7 @@ static struct cache_deferred_req *svc_defer(struct cache_req *req)
>>  	struct svc_rqst *rqstp = container_of(req, struct svc_rqst, rq_chandle);
>>  	struct svc_deferred_req *dr;
>>  
>> -	if (rqstp->rq_arg.page_len)
>> +	if (rqstp->rq_arg.page_len || !rqstp->rq_usedeferral)
>>  		return NULL; /* if more than a page, give up FIXME */
>>  	if (rqstp->rq_deferred) {
>>  		dr = rqstp->rq_deferred;
>> -- 
>> 1.6.2.1
>>

[PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Note: the NFSv4.1 client also uses (and declares) this pointer.
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/sunrpc/clnt.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index c39a210..928c547 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -53,6 +53,7 @@ struct rpc_clnt {
 	char			cl_pathname[30];/* Path in rpc_pipe_fs */
 	struct vfsmount *	cl_vfsmnt;
 	struct dentry *		cl_dentry;	/* inode */
+	void			*cl_private;	/* private data */
 	struct rpc_clnt *	cl_parent;	/* Points to parent of clones */
 	struct rpc_rtt		cl_rtt_default;
 	struct rpc_timeout	cl_timeout_default;
-- 
1.6.2.1

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[J. Bruce Fields]

On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Note: the NFSv4.1 client also uses (and declares) this pointer.

OK.  Ack from trond?

--b.

> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  include/linux/sunrpc/clnt.h |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
> index c39a210..928c547 100644
> --- a/include/linux/sunrpc/clnt.h
> +++ b/include/linux/sunrpc/clnt.h
> @@ -53,6 +53,7 @@ struct rpc_clnt {
>  	char			cl_pathname[30];/* Path in rpc_pipe_fs */
>  	struct vfsmount *	cl_vfsmnt;
>  	struct dentry *		cl_dentry;	/* inode */
> +	void			*cl_private;	/* private data */
>  	struct rpc_clnt *	cl_parent;	/* Points to parent of clones */
>  	struct rpc_rtt		cl_rtt_default;
>  	struct rpc_timeout	cl_timeout_default;
> -- 
> 1.6.2.1
> 

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Myklebust, Trond]

On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
wrote:

> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Note: the NFSv4.1 client also uses (and declares) this pointer.
>
> OK.  Ack from trond?


First, someone would need to remind me why it is necessary, and add  
that justification to the changelog.

     Trond
>
>
> --b.
>
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>> include/linux/sunrpc/clnt.h |    1 +
>> 1 files changed, 1 insertions(+), 0 deletions(-)
>>
>> diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/ 
>> clnt.h
>> index c39a210..928c547 100644
>> --- a/include/linux/sunrpc/clnt.h
>> +++ b/include/linux/sunrpc/clnt.h
>> @@ -53,6 +53,7 @@ struct rpc_clnt {
>>    char            cl_pathname[30];/* Path in rpc_pipe_fs */
>>    struct vfsmount *    cl_vfsmnt;
>>    struct dentry *        cl_dentry;    /* inode */
>> +    void            *cl_private;    /* private data */
>>    struct rpc_clnt *    cl_parent;    /* Points to parent of clones  
>> */
>>    struct rpc_rtt        cl_rtt_default;
>>    struct rpc_timeout    cl_timeout_default;
>> -- 
>> 1.6.2.1
>>

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Benny Halevy]

On Mar. 28, 2009, 3:39 +0300, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
> On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
> wrote:
> 
>> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
>>> From: Andy Adamson <andros@netapp.com>
>>>
>>> Note: the NFSv4.1 client also uses (and declares) this pointer.
>> OK.  Ack from trond?
> 
> 
> First, someone would need to remind me why it is necessary, and add  
> that justification to the changelog.

First time this is used in this patchset is here:
[PATCH 35/47] nfsd: minorversion support for the back channel

The client uses cl_private to determine the minorversion
(via a struct nfs_client *) to be set in the compound header,
and to know when to generate a SEQUENCE op.
Similarly, the server puts a struct nfs4_callback * in
there for callback compounds' CB_COMPOUND and CB_SEQUENCE.

Benny

> 
>      Trond
>>
>> --b.
>>
>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>> ---
>>> include/linux/sunrpc/clnt.h |    1 +
>>> 1 files changed, 1 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/ 
>>> clnt.h
>>> index c39a210..928c547 100644
>>> --- a/include/linux/sunrpc/clnt.h
>>> +++ b/include/linux/sunrpc/clnt.h
>>> @@ -53,6 +53,7 @@ struct rpc_clnt {
>>>    char            cl_pathname[30];/* Path in rpc_pipe_fs */
>>>    struct vfsmount *    cl_vfsmnt;
>>>    struct dentry *        cl_dentry;    /* inode */
>>> +    void            *cl_private;    /* private data */
>>>    struct rpc_clnt *    cl_parent;    /* Points to parent of clones  
>>> */
>>>    struct rpc_rtt        cl_rtt_default;
>>>    struct rpc_timeout    cl_timeout_default;
>>> -- 
>>> 1.6.2.1
>>>

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Trond Myklebust]

On Sat, 2009-03-28 at 11:20 +0300, Benny Halevy wrote:
> On Mar. 28, 2009, 3:39 +0300, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
> > On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
> > wrote:
> > 
> >> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
> >>> From: Andy Adamson <andros@netapp.com>
> >>>
> >>> Note: the NFSv4.1 client also uses (and declares) this pointer.
> >> OK.  Ack from trond?
> > 
> > 
> > First, someone would need to remind me why it is necessary, and add  
> > that justification to the changelog.
> 
> First time this is used in this patchset is here:
> [PATCH 35/47] nfsd: minorversion support for the back channel
> 
> The client uses cl_private to determine the minorversion
> (via a struct nfs_client *) to be set in the compound header,
> and to know when to generate a SEQUENCE op.
> Similarly, the server puts a struct nfs4_callback * in
> there for callback compounds' CB_COMPOUND and CB_SEQUENCE.

Why would the rpc_client need to know and track minor versions? That is
an NFS protocol specific thing.

Besides, the caller should always know what minor version it is using.
It shouldn't need a back-pointer in the rpc_client...

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Benny Halevy]

On Mar. 28, 2009, 20:23 +0300, Trond Myklebust <Trond.Myklebust@netapp.com> wrote:
> On Sat, 2009-03-28 at 11:20 +0300, Benny Halevy wrote:
>> On Mar. 28, 2009, 3:39 +0300, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>> On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
>>> wrote:
>>>
>>>> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
>>>>> From: Andy Adamson <andros@netapp.com>
>>>>>
>>>>> Note: the NFSv4.1 client also uses (and declares) this pointer.
>>>> OK.  Ack from trond?
>>>
>>> First, someone would need to remind me why it is necessary, and add  
>>> that justification to the changelog.
>> First time this is used in this patchset is here:
>> [PATCH 35/47] nfsd: minorversion support for the back channel
>>
>> The client uses cl_private to determine the minorversion
>> (via a struct nfs_client *) to be set in the compound header,
>> and to know when to generate a SEQUENCE op.
>> Similarly, the server puts a struct nfs4_callback * in
>> there for callback compounds' CB_COMPOUND and CB_SEQUENCE.
> 
> Why would the rpc_client need to know and track minor versions? That is
> an NFS protocol specific thing.

Agreed.  the rpc_client doesn't know anything about the minor version.
This is why we store it in a void *cl_private.

> 
> Besides, the caller should always know what minor version it is using.
> It shouldn't need a back-pointer in the rpc_client...
> 

The back pointer is used in the xdr layer like this:
 	struct xdr_stream xdr;
+	struct nfs_client *clp =
+		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.nops = 0,
+		.minorversion = clp->cl_minorversion,
 	};

Benny

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Trond Myklebust]

On Sat, 2009-03-28 at 20:29 +0300, Benny Halevy wrote:
> On Mar. 28, 2009, 20:23 +0300, Trond Myklebust <Trond.Myklebust@netapp.com> wrote:
> > On Sat, 2009-03-28 at 11:20 +0300, Benny Halevy wrote:
> >> On Mar. 28, 2009, 3:39 +0300, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
> >>> On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
> >>> wrote:
> >>>
> >>>> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
> >>>>> From: Andy Adamson <andros@netapp.com>
> >>>>>
> >>>>> Note: the NFSv4.1 client also uses (and declares) this pointer.
> >>>> OK.  Ack from trond?
> >>>
> >>> First, someone would need to remind me why it is necessary, and add  
> >>> that justification to the changelog.
> >> First time this is used in this patchset is here:
> >> [PATCH 35/47] nfsd: minorversion support for the back channel
> >>
> >> The client uses cl_private to determine the minorversion
> >> (via a struct nfs_client *) to be set in the compound header,
> >> and to know when to generate a SEQUENCE op.
> >> Similarly, the server puts a struct nfs4_callback * in
> >> there for callback compounds' CB_COMPOUND and CB_SEQUENCE.
> > 
> > Why would the rpc_client need to know and track minor versions? That is
> > an NFS protocol specific thing.
> 
> Agreed.  the rpc_client doesn't know anything about the minor version.
> This is why we store it in a void *cl_private.
> 
> > 
> > Besides, the caller should always know what minor version it is using.
> > It shouldn't need a back-pointer in the rpc_client...
> > 
> 
> The back pointer is used in the xdr layer like this:
>  	struct xdr_stream xdr;
> +	struct nfs_client *clp =
> +		(struct nfs_client *)req->rq_task->tk_client->cl_private;
>  	struct compound_hdr hdr = {
> -		.nops = 0,
> +		.minorversion = clp->cl_minorversion,
>  	};

Just put that information in the RPC arguments where it _BELONGS_.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Benny Halevy]

On Mar. 28, 2009, 20:35 +0300, Trond Myklebust <Trond.Myklebust@netapp.com> wrote:
> On Sat, 2009-03-28 at 20:29 +0300, Benny Halevy wrote:
>> On Mar. 28, 2009, 20:23 +0300, Trond Myklebust <Trond.Myklebust@netapp.com> wrote:
>>> On Sat, 2009-03-28 at 11:20 +0300, Benny Halevy wrote:
>>>> On Mar. 28, 2009, 3:39 +0300, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>> On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
>>>>> wrote:
>>>>>
>>>>>> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
>>>>>>> From: Andy Adamson <andros@netapp.com>
>>>>>>>
>>>>>>> Note: the NFSv4.1 client also uses (and declares) this pointer.
>>>>>> OK.  Ack from trond?
>>>>> First, someone would need to remind me why it is necessary, and add  
>>>>> that justification to the changelog.
>>>> First time this is used in this patchset is here:
>>>> [PATCH 35/47] nfsd: minorversion support for the back channel
>>>>
>>>> The client uses cl_private to determine the minorversion
>>>> (via a struct nfs_client *) to be set in the compound header,
>>>> and to know when to generate a SEQUENCE op.
>>>> Similarly, the server puts a struct nfs4_callback * in
>>>> there for callback compounds' CB_COMPOUND and CB_SEQUENCE.
>>> Why would the rpc_client need to know and track minor versions? That is
>>> an NFS protocol specific thing.
>> Agreed.  the rpc_client doesn't know anything about the minor version.
>> This is why we store it in a void *cl_private.
>>
>>> Besides, the caller should always know what minor version it is using.
>>> It shouldn't need a back-pointer in the rpc_client...
>>>
>> The back pointer is used in the xdr layer like this:
>>  	struct xdr_stream xdr;
>> +	struct nfs_client *clp =
>> +		(struct nfs_client *)req->rq_task->tk_client->cl_private;
>>  	struct compound_hdr hdr = {
>> -		.nops = 0,
>> +		.minorversion = clp->cl_minorversion,
>>  	};
> 
> Just put that information in the RPC arguments where it _BELONGS_.
> 

BTW, we are going back and forth on this...
cl_private was suggested on the review we did on November '08:
http://linux-nfs.org/pipermail/pnfs/2008-November/005435.html

Anyway, the following adds a *session to the nfs4_sequence_args
and res. It increases the memory footprint of all rpc args and res
but I agree it's cleaner.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
git diff --stat -p 2e00c20
 fs/nfs/client.c             |    1 -
 fs/nfs/nfs4proc.c           |    8 ++-
 fs/nfs/nfs4xdr.c            |  235 ++++++++++++++++---------------------------
 include/linux/nfs_xdr.h     |    7 ++
 include/linux/sunrpc/clnt.h |    1 -
 5 files changed, 99 insertions(+), 153 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 6e3ab63..bce02da 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -648,7 +648,6 @@ static int nfs_create_rpc_client(struct nfs_client *clp,
 	}
 
 	clp->cl_rpcclient = clnt;
-	clnt->cl_private = clp;
 	return 0;
 }
 
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 5614cfa..e997882 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -514,11 +514,13 @@ static int nfs41_setup_sequence(struct nfs4_session *session,
 	spin_unlock(&tbl->slot_tbl_lock);
 
 	slot = tbl->slots + slotid;
+	args->sa_session = session;
 	args->sa_slotid = slotid;
 	args->sa_cache_this = cache_reply;
 
 	dprintk("<-- %s slotid=%d seqid=%d\n", __func__, slotid, slot->seq_nr);
 
+	res->sr_session = session;
 	res->sr_slotid = slotid;
 	res->sr_renewal_time = jiffies;
 	/*
@@ -4244,6 +4246,7 @@ static int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 {
 	nfs4_verifier verifier;
 	struct nfs41_exchange_id_args args = {
+		.client = clp,
 		.flags = clp->cl_exchange_flags,
 	};
 	struct nfs41_exchange_id_res res = {
@@ -4659,10 +4662,13 @@ static int _nfs4_proc_create_session(struct nfs_client *clp)
 		.client = clp,
 		.cb_program = NFS4_CALLBACK,
 	};
-
+	struct nfs41_create_session_res res = {
+		.client = clp,
+	};
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_CREATE_SESSION],
 		.rpc_argp = &args,
+		.rpc_resp = &res,
 	};
 	int status;
 
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 342bb68..ae01d82 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1641,20 +1641,18 @@ static void encode_destroy_session(struct xdr_stream *xdr,
 #endif /* CONFIG_NFS_V4_1 */
 
 static void encode_sequence(struct xdr_stream *xdr,
-			    const struct nfs_client *clp,
 			    const struct nfs4_sequence_args *args,
 			    struct compound_hdr *hdr)
 {
 #if defined(CONFIG_NFS_V4_1)
-	struct nfs4_session *session;
+	struct nfs4_session *session = args->sa_session;
 	struct nfs4_slot_table *tp;
 	struct nfs4_slot *slot;
 	__be32 *p;
 
-	if (!clp->cl_minorversion)
+	if (!session)
 		return;
 
-	session = clp->cl_session;
 	tp = &session->fc_slot_table;
 
 	WARN_ON(args->sa_slotid == NFS4_MAX_SLOT_TABLE);
@@ -1690,21 +1688,28 @@ static void encode_sequence(struct xdr_stream *xdr,
  * END OF "GENERIC" ENCODE ROUTINES.
  */
 
+static u32 nfs4_xdr_minorversion(const struct nfs4_sequence_args *args)
+{
+#if defined(CONFIG_NFS_V4_1)
+	if (args->sa_session)
+		return args->sa_session->clp->cl_minorversion;
+#endif /* CONFIG_NFS_V4_1 */
+	return 0;
+}
+
 /*
  * Encode an ACCESS request
  */
 static int nfs4_xdr_enc_access(struct rpc_rqst *req, __be32 *p, const struct nfs4_accessargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_access(&xdr, args->access, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -1718,15 +1723,13 @@ static int nfs4_xdr_enc_access(struct rpc_rqst *req, __be32 *p, const struct nfs
 static int nfs4_xdr_enc_lookup(struct rpc_rqst *req, __be32 *p, const struct nfs4_lookup_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->dir_fh, &hdr);
 	encode_lookup(&xdr, args->name, &hdr);
 	encode_getfh(&xdr, &hdr);
@@ -1741,15 +1744,13 @@ static int nfs4_xdr_enc_lookup(struct rpc_rqst *req, __be32 *p, const struct nfs
 static int nfs4_xdr_enc_lookup_root(struct rpc_rqst *req, __be32 *p, const struct nfs4_lookup_root_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putrootfh(&xdr, &hdr);
 	encode_getfh(&xdr, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -1763,15 +1764,13 @@ static int nfs4_xdr_enc_lookup_root(struct rpc_rqst *req, __be32 *p, const struc
 static int nfs4_xdr_enc_remove(struct rpc_rqst *req, __be32 *p, const struct nfs_removeargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_remove(&xdr, &args->name, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -1785,15 +1784,13 @@ static int nfs4_xdr_enc_remove(struct rpc_rqst *req, __be32 *p, const struct nfs
 static int nfs4_xdr_enc_rename(struct rpc_rqst *req, __be32 *p, const struct nfs4_rename_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->old_dir, &hdr);
 	encode_savefh(&xdr, &hdr);
 	encode_putfh(&xdr, args->new_dir, &hdr);
@@ -1811,15 +1808,13 @@ static int nfs4_xdr_enc_rename(struct rpc_rqst *req, __be32 *p, const struct nfs
 static int nfs4_xdr_enc_link(struct rpc_rqst *req, __be32 *p, const struct nfs4_link_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_savefh(&xdr, &hdr);
 	encode_putfh(&xdr, args->dir_fh, &hdr);
@@ -1837,15 +1832,13 @@ static int nfs4_xdr_enc_link(struct rpc_rqst *req, __be32 *p, const struct nfs4_
 static int nfs4_xdr_enc_create(struct rpc_rqst *req, __be32 *p, const struct nfs4_create_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->dir_fh, &hdr);
 	encode_savefh(&xdr, &hdr);
 	encode_create(&xdr, args, &hdr);
@@ -1871,15 +1864,13 @@ static int nfs4_xdr_enc_symlink(struct rpc_rqst *req, __be32 *p, const struct nf
 static int nfs4_xdr_enc_getattr(struct rpc_rqst *req, __be32 *p, const struct nfs4_getattr_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
 	encode_nops(&hdr);
@@ -1892,15 +1883,13 @@ static int nfs4_xdr_enc_getattr(struct rpc_rqst *req, __be32 *p, const struct nf
 static int nfs4_xdr_enc_close(struct rpc_rqst *req, __be32 *p, struct nfs_closeargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_close(&xdr, args, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -1914,15 +1903,13 @@ static int nfs4_xdr_enc_close(struct rpc_rqst *req, __be32 *p, struct nfs_closea
 static int nfs4_xdr_enc_open(struct rpc_rqst *req, __be32 *p, struct nfs_openargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_savefh(&xdr, &hdr);
 	encode_open(&xdr, args, &hdr);
@@ -1958,15 +1945,13 @@ static int nfs4_xdr_enc_open_confirm(struct rpc_rqst *req, __be32 *p, struct nfs
 static int nfs4_xdr_enc_open_noattr(struct rpc_rqst *req, __be32 *p, struct nfs_openargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_open(&xdr, args, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -1980,15 +1965,13 @@ static int nfs4_xdr_enc_open_noattr(struct rpc_rqst *req, __be32 *p, struct nfs_
 static int nfs4_xdr_enc_open_downgrade(struct rpc_rqst *req, __be32 *p, struct nfs_closeargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_open_downgrade(&xdr, args, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -2002,15 +1985,13 @@ static int nfs4_xdr_enc_open_downgrade(struct rpc_rqst *req, __be32 *p, struct n
 static int nfs4_xdr_enc_lock(struct rpc_rqst *req, __be32 *p, struct nfs_lock_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_lock(&xdr, args, &hdr);
 	encode_nops(&hdr);
@@ -2023,15 +2004,13 @@ static int nfs4_xdr_enc_lock(struct rpc_rqst *req, __be32 *p, struct nfs_lock_ar
 static int nfs4_xdr_enc_lockt(struct rpc_rqst *req, __be32 *p, struct nfs_lockt_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_lockt(&xdr, args, &hdr);
 	encode_nops(&hdr);
@@ -2044,15 +2023,13 @@ static int nfs4_xdr_enc_lockt(struct rpc_rqst *req, __be32 *p, struct nfs_lockt_
 static int nfs4_xdr_enc_locku(struct rpc_rqst *req, __be32 *p, struct nfs_locku_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_locku(&xdr, args, &hdr);
 	encode_nops(&hdr);
@@ -2065,15 +2042,13 @@ static int nfs4_xdr_enc_locku(struct rpc_rqst *req, __be32 *p, struct nfs_locku_
 static int nfs4_xdr_enc_readlink(struct rpc_rqst *req, __be32 *p, const struct nfs4_readlink *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_readlink(&xdr, args, req, &hdr);
 
@@ -2089,15 +2064,13 @@ static int nfs4_xdr_enc_readlink(struct rpc_rqst *req, __be32 *p, const struct n
 static int nfs4_xdr_enc_readdir(struct rpc_rqst *req, __be32 *p, const struct nfs4_readdir_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_readdir(&xdr, args, req, &hdr);
 
@@ -2116,15 +2089,13 @@ static int nfs4_xdr_enc_readdir(struct rpc_rqst *req, __be32 *p, const struct nf
 static int nfs4_xdr_enc_read(struct rpc_rqst *req, __be32 *p, struct nfs_readargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_read(&xdr, args, &hdr);
 
@@ -2141,15 +2112,13 @@ static int nfs4_xdr_enc_read(struct rpc_rqst *req, __be32 *p, struct nfs_readarg
 static int nfs4_xdr_enc_setattr(struct rpc_rqst *req, __be32 *p, struct nfs_setattrargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_setattr(&xdr, args, args->server, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -2165,15 +2134,13 @@ nfs4_xdr_enc_getacl(struct rpc_rqst *req, __be32 *p,
 		struct nfs_getaclargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_getattr_two(&xdr, FATTR4_WORD0_ACL, 0, &hdr);
 
@@ -2189,15 +2156,13 @@ nfs4_xdr_enc_getacl(struct rpc_rqst *req, __be32 *p,
 static int nfs4_xdr_enc_write(struct rpc_rqst *req, __be32 *p, struct nfs_writeargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_write(&xdr, args, &hdr);
 	req->rq_snd_buf.flags |= XDRBUF_WRITE;
@@ -2212,15 +2177,13 @@ static int nfs4_xdr_enc_write(struct rpc_rqst *req, __be32 *p, struct nfs_writea
 static int nfs4_xdr_enc_commit(struct rpc_rqst *req, __be32 *p, struct nfs_writeargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_commit(&xdr, args, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -2234,15 +2197,13 @@ static int nfs4_xdr_enc_commit(struct rpc_rqst *req, __be32 *p, struct nfs_write
 static int nfs4_xdr_enc_fsinfo(struct rpc_rqst *req, __be32 *p, struct nfs4_fsinfo_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_fsinfo(&xdr, args->bitmask, &hdr);
 	encode_nops(&hdr);
@@ -2255,15 +2216,13 @@ static int nfs4_xdr_enc_fsinfo(struct rpc_rqst *req, __be32 *p, struct nfs4_fsin
 static int nfs4_xdr_enc_pathconf(struct rpc_rqst *req, __be32 *p, const struct nfs4_pathconf_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_getattr_one(&xdr, args->bitmask[0] & nfs4_pathconf_bitmap[0],
 			   &hdr);
@@ -2277,15 +2236,13 @@ static int nfs4_xdr_enc_pathconf(struct rpc_rqst *req, __be32 *p, const struct n
 static int nfs4_xdr_enc_statfs(struct rpc_rqst *req, __be32 *p, const struct nfs4_statfs_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	encode_getattr_two(&xdr, args->bitmask[0] & nfs4_statfs_bitmap[0],
 			   args->bitmask[1] & nfs4_statfs_bitmap[1], &hdr);
@@ -2300,15 +2257,13 @@ static int nfs4_xdr_enc_server_caps(struct rpc_rqst *req, __be32 *p,
 				    struct nfs4_server_caps_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fhandle, &hdr);
 	encode_getattr_one(&xdr, FATTR4_WORD0_SUPPORTED_ATTRS|
 			   FATTR4_WORD0_LINK_SUPPORT|
@@ -2378,15 +2333,13 @@ static int nfs4_xdr_enc_setclientid_confirm(struct rpc_rqst *req, __be32 *p, str
 static int nfs4_xdr_enc_delegreturn(struct rpc_rqst *req, __be32 *p, const struct nfs4_delegreturnargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fhandle, &hdr);
 	encode_delegreturn(&xdr, args->stateid, &hdr);
 	encode_getfattr(&xdr, args->bitmask, &hdr);
@@ -2400,15 +2353,13 @@ static int nfs4_xdr_enc_delegreturn(struct rpc_rqst *req, __be32 *p, const struc
 static int nfs4_xdr_enc_fs_locations(struct rpc_rqst *req, __be32 *p, struct nfs4_fs_locations_arg *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->dir_fh, &hdr);
 	encode_lookup(&xdr, args->name, &hdr);
 	encode_fs_locations(&xdr, args->bitmask, &hdr);
@@ -2424,13 +2375,11 @@ static int nfs4_xdr_enc_fs_locations(struct rpc_rqst *req, __be32 *p, struct nfs
  * EXCHANGE_ID request
  */
 static int nfs4_xdr_enc_exchange_id(struct rpc_rqst *req, uint32_t *p,
-				    void *args)
+				    struct nfs41_exchange_id_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = args->client->cl_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
@@ -2447,10 +2396,8 @@ static int nfs4_xdr_enc_create_session(struct rpc_rqst *req, uint32_t *p,
 				       struct nfs41_create_session_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = args->client->cl_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
@@ -2467,10 +2414,8 @@ static int nfs4_xdr_enc_destroy_session(struct rpc_rqst *req, uint32_t *p,
 					struct nfs4_session *session)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = session->clp->cl_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
@@ -2487,15 +2432,13 @@ static int nfs4_xdr_enc_sequence(struct rpc_rqst *req, uint32_t *p,
 				 struct nfs4_sequence_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(args),
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, args, &hdr);
+	encode_sequence(&xdr, args, &hdr);
 	encode_nops(&hdr);
 	return 0;
 }
@@ -2507,16 +2450,14 @@ static int nfs4_xdr_enc_get_lease_time(struct rpc_rqst *req, uint32_t *p,
 				       struct nfs4_get_lease_time_args *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->la_seq_args),
 	};
 	const u32 lease_bitmap[2] = { FATTR4_WORD0_LEASE_TIME, 0 };
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->la_seq_args, &hdr);
+	encode_sequence(&xdr, &args->la_seq_args, &hdr);
 	encode_putrootfh(&xdr, &hdr);
 	encode_fsinfo(&xdr, lease_bitmap, &hdr);
 	encode_nops(&hdr);
@@ -4198,13 +4139,13 @@ static int decode_exchange_id(struct xdr_stream *xdr,
 	return 0;
 }
 
-static int decode_create_session(struct xdr_stream *xdr, struct rpc_rqst *rqstp)
+static int decode_create_session(struct xdr_stream *xdr,
+				 struct nfs41_create_session_res *res)
 {
 	uint32_t *p;
 	int status;
 	u32 nr_attrs;
-	struct nfs_client *clp =
-		(struct nfs_client *)rqstp->rq_task->tk_client->cl_private;
+	struct nfs_client *clp = res->client;
 	struct nfs4_session *session = clp->cl_session;
 
 	status = decode_op_hdr(xdr, OP_CREATE_SESSION);
@@ -4266,16 +4207,13 @@ static int decode_sequence(struct xdr_stream *xdr,
 			   struct rpc_rqst *rqstp)
 {
 #if defined(CONFIG_NFS_V4_1)
-	struct nfs4_slot_table *tp;
 	struct nfs4_slot *slot;
-	struct nfs_client *clp =
-		(struct nfs_client *)rqstp->rq_task->tk_client->cl_private;
 	struct nfs4_sessionid id;
 	u32 dummy;
 	int status;
 	__be32 *p;
 
-	if (clp->cl_minorversion == 0)
+	if (!res->sr_session)
 		return 0;
 
 	status = decode_op_hdr(xdr, OP_SEQUENCE);
@@ -4288,11 +4226,10 @@ static int decode_sequence(struct xdr_stream *xdr,
 	 */
 	status = -ESERVERFAULT;
 
-	tp = &clp->cl_session->fc_slot_table;
-	slot = tp->slots + res->sr_slotid;
+	slot = &res->sr_session->fc_slot_table.slots[res->sr_slotid];
 	READ_BUF(NFS4_MAX_SESSIONID_LEN + 20);
 	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
-	if (memcmp(id.data, clp->cl_session->sess_id.data,
+	if (memcmp(id.data, res->sr_session->sess_id.data,
 		   NFS4_MAX_SESSIONID_LEN)) {
 		dprintk("%s Invalid session id\n", __func__);
 		goto out_err;
@@ -4601,16 +4538,14 @@ static int
 nfs4_xdr_enc_setacl(struct rpc_rqst *req, __be32 *p, struct nfs_setaclargs *args)
 {
 	struct xdr_stream xdr;
-	struct nfs_client *clp =
-		(struct nfs_client *)req->rq_task->tk_client->cl_private;
 	struct compound_hdr hdr = {
-		.minorversion = clp->cl_minorversion,
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
 	};
 	int status;
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_compound_hdr(&xdr, req, &hdr);
-	encode_sequence(&xdr, clp, &args->seq_args, &hdr);
+	encode_sequence(&xdr, &args->seq_args, &hdr);
 	encode_putfh(&xdr, args->fh, &hdr);
 	status = encode_setacl(&xdr, args, &hdr);
 	encode_nops(&hdr);
@@ -5233,7 +5168,7 @@ static int nfs4_xdr_dec_exchange_id(struct rpc_rqst *rqstp, uint32_t *p,
  * a CREATE_SESSION request
  */
 static int nfs4_xdr_dec_create_session(struct rpc_rqst *rqstp, uint32_t *p,
-				       void *res)
+				       struct nfs41_create_session_res *res)
 {
 	struct xdr_stream xdr;
 	struct compound_hdr hdr;
@@ -5242,7 +5177,7 @@ static int nfs4_xdr_dec_create_session(struct rpc_rqst *rqstp, uint32_t *p,
 	xdr_init_decode(&xdr, &rqstp->rq_rcv_buf, p);
 	status = decode_compound_hdr(&xdr, &hdr);
 	if (!status)
-		status = decode_create_session(&xdr, rqstp);
+		status = decode_create_session(&xdr, res);
 	return status;
 }
 
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 517291e..4feacc6 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -130,11 +130,13 @@ struct nfs4_slot {
 };
 
 struct nfs4_sequence_args {
+	struct nfs4_session	*sa_session;
 	u8			sa_slotid;
 	u8			sa_cache_this;
 };
 
 struct nfs4_sequence_res {
+	struct nfs4_session	*sr_session;
 	u8			sr_slotid;	/* slot used to send request */
 	unsigned long		sr_renewal_time;
 	int			sr_status;	/* sequence operation status */
@@ -870,6 +872,7 @@ struct nfs_impl_id4 {
 
 #define NFS4_EXCHANGE_ID_LEN	(48)
 struct nfs41_exchange_id_args {
+	struct nfs_client		*client;
 	nfs4_verifier			*verifier;
 	unsigned int 			id_len;
 	char 				id[NFS4_EXCHANGE_ID_LEN];
@@ -902,6 +905,10 @@ struct nfs41_create_session_args {
 	struct nfs4_channel_attrs	fc_attrs;	/* Fore Channel */
 	struct nfs4_channel_attrs	bc_attrs;	/* Back Channel */
 };
+
+struct nfs41_create_session_res {
+	struct nfs_client	       *client;
+};
 #endif /* CONFIG_NFS_V4_1 */
 
 struct nfs_page;
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 8ca3fef..37881f1 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -53,7 +53,6 @@ struct rpc_clnt {
 	char			cl_pathname[30];/* Path in rpc_pipe_fs */
 	struct vfsmount *	cl_vfsmnt;
 	struct dentry *		cl_dentry;	/* inode */
-	void			*cl_private;	/* private data */
 	struct rpc_clnt *	cl_parent;	/* Points to parent of clones */
 	struct rpc_rtt		cl_rtt_default;
 	struct rpc_timeout	cl_timeout_default;

Re: [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt

[Trond Myklebust]

On Sat, 2009-03-28 at 22:26 +0300, Benny Halevy wrote:
> On Mar. 28, 2009, 20:35 +0300, Trond Myklebust <Trond.Myklebust@netapp.com> wrote:
> > On Sat, 2009-03-28 at 20:29 +0300, Benny Halevy wrote:
> >> On Mar. 28, 2009, 20:23 +0300, Trond Myklebust <Trond.Myklebust@netapp.com> wrote:
> >>> On Sat, 2009-03-28 at 11:20 +0300, Benny Halevy wrote:
> >>>> On Mar. 28, 2009, 3:39 +0300, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
> >>>>> On Mar 27, 2009, at 8:06 PM, "J. Bruce Fields" <bfields@fieldses.org>  
> >>>>> wrote:
> >>>>>
> >>>>>> On Fri, Mar 27, 2009 at 06:01:48AM +0300, Benny Halevy wrote:
> >>>>>>> From: Andy Adamson <andros@netapp.com>
> >>>>>>>
> >>>>>>> Note: the NFSv4.1 client also uses (and declares) this pointer.
> >>>>>> OK.  Ack from trond?
> >>>>> First, someone would need to remind me why it is necessary, and add  
> >>>>> that justification to the changelog.
> >>>> First time this is used in this patchset is here:
> >>>> [PATCH 35/47] nfsd: minorversion support for the back channel
> >>>>
> >>>> The client uses cl_private to determine the minorversion
> >>>> (via a struct nfs_client *) to be set in the compound header,
> >>>> and to know when to generate a SEQUENCE op.
> >>>> Similarly, the server puts a struct nfs4_callback * in
> >>>> there for callback compounds' CB_COMPOUND and CB_SEQUENCE.
> >>> Why would the rpc_client need to know and track minor versions? That is
> >>> an NFS protocol specific thing.
> >> Agreed.  the rpc_client doesn't know anything about the minor version.
> >> This is why we store it in a void *cl_private.
> >>
> >>> Besides, the caller should always know what minor version it is using.
> >>> It shouldn't need a back-pointer in the rpc_client...
> >>>
> >> The back pointer is used in the xdr layer like this:
> >>  	struct xdr_stream xdr;
> >> +	struct nfs_client *clp =
> >> +		(struct nfs_client *)req->rq_task->tk_client->cl_private;
> >>  	struct compound_hdr hdr = {
> >> -		.nops = 0,
> >> +		.minorversion = clp->cl_minorversion,
> >>  	};
> > 
> > Just put that information in the RPC arguments where it _BELONGS_.
> > 
> 
> BTW, we are going back and forth on this...
> cl_private was suggested on the review we did on November '08:
> http://linux-nfs.org/pipermail/pnfs/2008-November/005435.html

No. I've never acked the idea of putting RPC function arguments into
private fields in the rpc_client. That's simply unacceptably ugly. If
you and Andy got that impression, then one of us must have misunderstood
what the other party was saying.

I do seem to remember talk about needing back-pointers to the session in
order to support the NFSv4.1 backchannel, but that's an entirely
different matter.

Note that it is probably worth caching the minor version inside the
session pointer instead of having to dereference the nfs_client on every
RPC call.
That can be a separate patch, though.

> Anyway, the following adds a *session to the nfs4_sequence_args
> and res. It increases the memory footprint of all rpc args and res
> but I agree it's cleaner.

The session is clearly an argument to the sequence op, and so deserves
to be kept in the rpc_args.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

[PATCH 03/47] nfsd: embed nfsd4_current_state in nfsd4_compoundres

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Remove the allocation of struct nfsd4_compound_state.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   40 +++++++++-------------------------------
 include/linux/nfsd/xdr4.h |    9 +++++----
 2 files changed, 14 insertions(+), 35 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 7839654..c8e1ad0 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -809,29 +809,6 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
 		nfsdstats.nfs4_opcount[opnum]++;
 }
 
-static void cstate_free(struct nfsd4_compound_state *cstate)
-{
-	if (cstate == NULL)
-		return;
-	fh_put(&cstate->current_fh);
-	fh_put(&cstate->save_fh);
-	BUG_ON(cstate->replay_owner);
-	kfree(cstate);
-}
-
-static struct nfsd4_compound_state *cstate_alloc(void)
-{
-	struct nfsd4_compound_state *cstate;
-
-	cstate = kmalloc(sizeof(struct nfsd4_compound_state), GFP_KERNEL);
-	if (cstate == NULL)
-		return NULL;
-	fh_init(&cstate->current_fh, NFS4_FHSIZE);
-	fh_init(&cstate->save_fh, NFS4_FHSIZE);
-	cstate->replay_owner = NULL;
-	return cstate;
-}
-
 typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
 			      void *);
 
@@ -859,12 +836,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 {
 	struct nfsd4_op	*op;
 	struct nfsd4_operation *opdesc;
-	struct nfsd4_compound_state *cstate = NULL;
+	struct nfsd4_compound_state *cstate = &resp->cstate;
 	int		slack_bytes;
 	__be32		status;
 
 	resp->xbuf = &rqstp->rq_res;
-	resp->p = rqstp->rq_res.head[0].iov_base + rqstp->rq_res.head[0].iov_len;
+	resp->p = rqstp->rq_res.head[0].iov_base +
+						rqstp->rq_res.head[0].iov_len;
 	resp->tagp = resp->p;
 	/* reserve space for: taglen, tag, and opcnt */
 	resp->p += 2 + XDR_QUADLEN(args->taglen);
@@ -873,6 +851,9 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	resp->tag = args->tag;
 	resp->opcnt = 0;
 	resp->rqstp = rqstp;
+	resp->cstate.replay_owner = NULL;
+	fh_init(&resp->cstate.current_fh, NFS4_FHSIZE);
+	fh_init(&resp->cstate.save_fh, NFS4_FHSIZE);
 	/* Use the deferral mechanism only for NFSv4.0 compounds */
 	rqstp->rq_usedeferral = (args->minorversion == 0);
 
@@ -883,11 +864,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
 		goto out;
 
-	status = nfserr_resource;
-	cstate = cstate_alloc();
-	if (cstate == NULL)
-		goto out;
-
 	status = nfs_ok;
 	while (!status && resp->opcnt < args->opcnt) {
 		op = &args->ops[resp->opcnt++];
@@ -964,7 +940,9 @@ encode_op:
 		status = nfserr_jukebox;
 	}
 
-	cstate_free(cstate);
+	fh_put(&resp->cstate.current_fh);
+	fh_put(&resp->cstate.save_fh);
+	BUG_ON(resp->cstate.replay_owner);
 out:
 	nfsd4_release_compoundargs(args);
 	/* Reset deferral mechanism for RPC deferrals */
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 27bd3e3..fd15ddc 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -45,9 +45,9 @@
 #define XDR_LEN(n)                     (((n) + 3) & ~3)
 
 struct nfsd4_compound_state {
-	struct svc_fh current_fh;
-	struct svc_fh save_fh;
-	struct nfs4_stateowner *replay_owner;
+	struct svc_fh		current_fh;
+	struct svc_fh		save_fh;
+	struct nfs4_stateowner	*replay_owner;
 };
 
 struct nfsd4_change_info {
@@ -416,7 +416,8 @@ struct nfsd4_compoundres {
 	u32				taglen;
 	char *				tag;
 	u32				opcnt;
-	__be32 *			tagp; /* where to encode tag and  opcount */
+	__be32 *			tagp; /* tag, opcount encode location */
+	struct nfsd4_compound_state	cstate;
 };
 
 #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
-- 
1.6.2.1

[PATCH 04/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

To be used for sessions DRC replay.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfsd/xdr4.h |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index fd15ddc..e309d55 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -48,6 +48,8 @@ struct nfsd4_compound_state {
 	struct svc_fh		current_fh;
 	struct svc_fh		save_fh;
 	struct nfs4_stateowner	*replay_owner;
+	/* For sessions DRC */
+	struct nfsd4_slot	*slot;
 };
 
 struct nfsd4_change_info {
-- 
1.6.2.1

[PATCH 05/47] nfs41: common protocol definitions

[Benny Halevy]

Define all NFSv4.1 common operation and error code constants.

Note that some of the definitions are used by both the nfs41 client
and the server code. This patch is duplicated in the nfs41 and nfsd41
sessions patchset.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfs41: add exchange id flags]
Signed-off-by: Mike Sager <sager@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[removed server-only hunk changing NFSERR_REPLAY_ME]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfs41: add SEQ4_XX to nfs41-common-protocol]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfs41: generic error code update]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfs4.h |  128 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 127 insertions(+), 1 deletions(-)

diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index b912311..a34683e 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -21,6 +21,7 @@
 #define NFS4_FHSIZE		128
 #define NFS4_MAXPATHLEN		PATH_MAX
 #define NFS4_MAXNAMLEN		NAME_MAX
+#define NFS4_MAX_SESSIONID_LEN	16
 
 #define NFS4_ACCESS_READ        0x0001
 #define NFS4_ACCESS_LOOKUP      0x0002
@@ -38,6 +39,7 @@
 #define NFS4_OPEN_RESULT_CONFIRM 0x0002
 #define NFS4_OPEN_RESULT_LOCKTYPE_POSIX 0x0004
 
+#define NFS4_SHARE_ACCESS_MASK	0x000F
 #define NFS4_SHARE_ACCESS_READ	0x0001
 #define NFS4_SHARE_ACCESS_WRITE	0x0002
 #define NFS4_SHARE_ACCESS_BOTH	0x0003
@@ -45,6 +47,19 @@
 #define NFS4_SHARE_DENY_WRITE	0x0002
 #define NFS4_SHARE_DENY_BOTH	0x0003
 
+/* nfs41 */
+#define NFS4_SHARE_WANT_MASK		0xFF00
+#define NFS4_SHARE_WANT_NO_PREFERENCE	0x0000
+#define NFS4_SHARE_WANT_READ_DELEG	0x0100
+#define NFS4_SHARE_WANT_WRITE_DELEG	0x0200
+#define NFS4_SHARE_WANT_ANY_DELEG	0x0300
+#define NFS4_SHARE_WANT_NO_DELEG	0x0400
+#define NFS4_SHARE_WANT_CANCEL		0x0500
+
+#define NFS4_SHARE_WHEN_MASK		0xF0000
+#define NFS4_SHARE_SIGNAL_DELEG_WHEN_RESRC_AVAIL	0x10000
+#define NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED		0x20000
+
 #define NFS4_SET_TO_SERVER_TIME	0
 #define NFS4_SET_TO_CLIENT_TIME	1
 
@@ -88,6 +103,31 @@
 #define NFS4_ACE_GENERIC_EXECUTE              0x001200A0
 #define NFS4_ACE_MASK_ALL                     0x001F01FF
 
+#define EXCHGID4_FLAG_SUPP_MOVED_REFER		0x00000001
+#define EXCHGID4_FLAG_SUPP_MOVED_MIGR		0x00000002
+#define EXCHGID4_FLAG_USE_NON_PNFS		0x00010000
+#define EXCHGID4_FLAG_USE_PNFS_MDS		0x00020000
+#define EXCHGID4_FLAG_USE_PNFS_DS		0x00040000
+#define EXCHGID4_FLAG_UPD_CONFIRMED_REC_A	0x40000000
+#define EXCHGID4_FLAG_CONFIRMED_R		0x80000000
+/*
+ * Since the validity of these bits depends on whether
+ * they're set in the argument or response, have separate
+ * invalid flag masks for arg (_A) and resp (_R).
+ */
+#define EXCHGID4_INVAL_FLAG_MASK_A		0xBFF8FFFC
+#define EXCHGID4_INVAL_FLAG_MASK_R		0x7FF8FFFC
+
+#define SEQ4_STATUS_CB_PATH_DOWN		0x00000001
+#define SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRING	0x00000002
+#define SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRED	0x00000004
+#define SEQ4_STATUS_EXPIRED_ALL_STATE_REVOKED	0x00000008
+#define SEQ4_STATUS_EXPIRED_SOME_STATE_REVOKED	0x00000010
+#define SEQ4_STATUS_ADMIN_STATE_REVOKED		0x00000020
+#define SEQ4_STATUS_RECALLABLE_STATE_REVOKED	0x00000040
+#define SEQ4_STATUS_LEASE_MOVED			0x00000080
+#define SEQ4_STATUS_RESTART_RECLAIM_NEEDED	0x00000100
+
 #define NFS4_MAX_UINT64	(~(u64)0)
 
 enum nfs4_acl_whotype {
@@ -154,6 +194,28 @@ enum nfs_opnum4 {
 	OP_VERIFY = 37,
 	OP_WRITE = 38,
 	OP_RELEASE_LOCKOWNER = 39,
+
+	/* nfs41 */
+	OP_BACKCHANNEL_CTL = 40,
+	OP_BIND_CONN_TO_SESSION = 41,
+	OP_EXCHANGE_ID = 42,
+	OP_CREATE_SESSION = 43,
+	OP_DESTROY_SESSION = 44,
+	OP_FREE_STATEID = 45,
+	OP_GET_DIR_DELEGATION = 46,
+	OP_GETDEVICEINFO = 47,
+	OP_GETDEVICELIST = 48,
+	OP_LAYOUTCOMMIT = 49,
+	OP_LAYOUTGET = 50,
+	OP_LAYOUTRETURN = 51,
+	OP_SECINFO_NO_NAME = 52,
+	OP_SEQUENCE = 53,
+	OP_SET_SSV = 54,
+	OP_TEST_STATEID = 55,
+	OP_WANT_DELEGATION = 56,
+	OP_DESTROY_CLIENTID = 57,
+	OP_RECLAIM_COMPLETE = 58,
+
 	OP_ILLEGAL = 10044,
 };
 
@@ -230,7 +292,48 @@ enum nfsstat4 {
 	NFS4ERR_DEADLOCK = 10045,
 	NFS4ERR_FILE_OPEN = 10046,
 	NFS4ERR_ADMIN_REVOKED = 10047,
-	NFS4ERR_CB_PATH_DOWN = 10048
+	NFS4ERR_CB_PATH_DOWN = 10048,
+
+	/* nfs41 */
+	NFS4ERR_BADIOMODE	= 10049,
+	NFS4ERR_BADLAYOUT	= 10050,
+	NFS4ERR_BAD_SESSION_DIGEST = 10051,
+	NFS4ERR_BADSESSION	= 10052,
+	NFS4ERR_BADSLOT		= 10053,
+	NFS4ERR_COMPLETE_ALREADY = 10054,
+	NFS4ERR_CONN_NOT_BOUND_TO_SESSION = 10055,
+	NFS4ERR_DELEG_ALREADY_WANTED = 10056,
+	NFS4ERR_BACK_CHAN_BUSY	= 10057,	/* backchan reqs outstanding */
+	NFS4ERR_LAYOUTTRYLATER	= 10058,
+	NFS4ERR_LAYOUTUNAVAILABLE = 10059,
+	NFS4ERR_NOMATCHING_LAYOUT = 10060,
+	NFS4ERR_RECALLCONFLICT	= 10061,
+	NFS4ERR_UNKNOWN_LAYOUTTYPE = 10062,
+	NFS4ERR_SEQ_MISORDERED = 10063, 	/* unexpected seq.id in req */
+	NFS4ERR_SEQUENCE_POS	= 10064,	/* [CB_]SEQ. op not 1st op */
+	NFS4ERR_REQ_TOO_BIG	= 10065,	/* request too big */
+	NFS4ERR_REP_TOO_BIG	= 10066,	/* reply too big */
+	NFS4ERR_REP_TOO_BIG_TO_CACHE = 10067,	/* rep. not all cached */
+	NFS4ERR_RETRY_UNCACHED_REP = 10068,	/* retry & rep. uncached */
+	NFS4ERR_UNSAFE_COMPOUND = 10069,	/* retry/recovery too hard */
+	NFS4ERR_TOO_MANY_OPS	= 10070,	/* too many ops in [CB_]COMP */
+	NFS4ERR_OP_NOT_IN_SESSION = 10071,	/* op needs [CB_]SEQ. op */
+	NFS4ERR_HASH_ALG_UNSUPP = 10072,	/* hash alg. not supp. */
+						/* Error 10073 is unused. */
+	NFS4ERR_CLIENTID_BUSY	= 10074,	/* clientid has state */
+	NFS4ERR_PNFS_IO_HOLE	= 10075,	/* IO to _SPARSE file hole */
+	NFS4ERR_SEQ_FALSE_RETRY	= 10076,	/* retry not origional */
+	NFS4ERR_BAD_HIGH_SLOT	= 10077,	/* sequence arg bad */
+	NFS4ERR_DEADSESSION	= 10078,	/* persistent session dead */
+	NFS4ERR_ENCR_ALG_UNSUPP = 10079,	/* SSV alg mismatch */
+	NFS4ERR_PNFS_NO_LAYOUT	= 10080,	/* direct I/O with no layout */
+	NFS4ERR_NOT_ONLY_OP	= 10081,	/* bad compound */
+	NFS4ERR_WRONG_CRED	= 10082,	/* permissions:state change */
+	NFS4ERR_WRONG_TYPE	= 10083,	/* current operation mismatch */
+	NFS4ERR_DIRDELEG_UNAVAIL = 10084,	/* no directory delegation */
+	NFS4ERR_REJECT_DELEG	= 10085,	/* on callback */
+	NFS4ERR_RETURNCONFLICT	= 10086,	/* outstanding layoutreturn */
+	NFS4ERR_DELEG_REVOKED	= 10087,	/* deleg./layout revoked */
 };
 
 /*
@@ -391,6 +494,29 @@ enum {
 	NFSPROC4_CLNT_GETACL,
 	NFSPROC4_CLNT_SETACL,
 	NFSPROC4_CLNT_FS_LOCATIONS,
+
+	/* nfs41 */
+	NFSPROC4_CLNT_EXCHANGE_ID,
+	NFSPROC4_CLNT_CREATE_SESSION,
+	NFSPROC4_CLNT_DESTROY_SESSION,
+	NFSPROC4_CLNT_SEQUENCE,
+	NFSPROC4_CLNT_GET_LEASE_TIME,
+};
+
+/* nfs41 types */
+struct nfs4_sessionid {
+	unsigned char data[NFS4_MAX_SESSIONID_LEN];
+};
+
+/* Create Session Flags */
+#define SESSION4_PERSIST	 0x001
+#define SESSION4_BACK_CHAN 	 0x002
+#define SESSION4_RDMA		 0x004
+
+enum state_protect_how4 {
+	SP4_NONE	= 0,
+	SP4_MACH_CRED	= 1,
+	SP4_SSV		= 2
 };
 
 #endif
-- 
1.6.2.1

[PATCH 06/47] nfsd41: change NFSERR_REPLAY_ME

[Benny Halevy]

so not to overlap nfs41 error codes.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfs.h |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/include/linux/nfs.h b/include/linux/nfs.h
index 54af92c..1510a64 100644
--- a/include/linux/nfs.h
+++ b/include/linux/nfs.h
@@ -109,7 +109,8 @@
 	NFSERR_FILE_OPEN = 10046,      /*       v4 */
 	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
 	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
-	NFSERR_REPLAY_ME = 10049	/*       v4 */
+
+	NFSERR_REPLAY_ME = 11001,	/* linux internal */
 };
 
 /* NFSv2 file types - beware, these are not the same in NFSv3 */
-- 
1.6.2.1

[PATCH 07/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment

At the moment we're expecting folks trying out nfs41 to
actively participate in the development process by helping us
debug issues and ideally send patches to fix problems.

[trailing space fixed]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/Kconfig |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/Kconfig b/fs/nfsd/Kconfig
index 503b9da..1f2f893 100644
--- a/fs/nfsd/Kconfig
+++ b/fs/nfsd/Kconfig
@@ -79,3 +79,12 @@ config NFSD_V4
 	  available from http://linux-nfs.org/.
 
 	  If unsure, say N.
+
+config NFSD_V4_1
+	bool "NFS server support for NFSv4.1 (DEVELOPER ONLY)"
+	depends on NFSD_V4 && EXPERIMENTAL
+	help
+	  This option enables support for minor version 1 of the NFSv4 protocol
+	  (draft-ietf-nfsv4-minorversion1) in the kernel's NFS server.
+
+	  Unless you're an NFS developer, say N.
-- 
1.6.2.1

[PATCH 08/47] nfsd41: define nfs41 error codes

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Define all error code present in
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: clean up error code definitions]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfs.h       |    2 --
 include/linux/nfsd/nfsd.h |   43 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/include/linux/nfs.h b/include/linux/nfs.h
index 1510a64..214d499 100644
--- a/include/linux/nfs.h
+++ b/include/linux/nfs.h
@@ -109,8 +109,6 @@
 	NFSERR_FILE_OPEN = 10046,      /*       v4 */
 	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
 	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
-
-	NFSERR_REPLAY_ME = 11001,	/* linux internal */
 };
 
 /* NFSv2 file types - beware, these are not the same in NFSv3 */
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index 54beda1..ab9616d 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -249,7 +249,44 @@ void		nfsd_lockd_shutdown(void);
 #define	nfserr_cb_path_down	cpu_to_be32(NFSERR_CB_PATH_DOWN)
 #define	nfserr_locked		cpu_to_be32(NFSERR_LOCKED)
 #define	nfserr_wrongsec		cpu_to_be32(NFSERR_WRONGSEC)
-#define	nfserr_replay_me	cpu_to_be32(NFSERR_REPLAY_ME)
+#define nfserr_badiomode		cpu_to_be32(NFS4ERR_BADIOMODE)
+#define nfserr_badlayout		cpu_to_be32(NFS4ERR_BADLAYOUT)
+#define nfserr_bad_session_digest	cpu_to_be32(NFS4ERR_BAD_SESSION_DIGEST)
+#define nfserr_badsession		cpu_to_be32(NFS4ERR_BADSESSION)
+#define nfserr_badslot			cpu_to_be32(NFS4ERR_BADSLOT)
+#define nfserr_complete_already		cpu_to_be32(NFS4ERR_COMPLETE_ALREADY)
+#define nfserr_conn_not_bound_to_session cpu_to_be32(NFS4ERR_CONN_NOT_BOUND_TO_SESSION)
+#define nfserr_deleg_already_wanted	cpu_to_be32(NFS4ERR_DELEG_ALREADY_WANTED)
+#define nfserr_back_chan_busy		cpu_to_be32(NFS4ERR_BACK_CHAN_BUSY)
+#define nfserr_layouttrylater		cpu_to_be32(NFS4ERR_LAYOUTTRYLATER)
+#define nfserr_layoutunavailable	cpu_to_be32(NFS4ERR_LAYOUTUNAVAILABLE)
+#define nfserr_nomatching_layout	cpu_to_be32(NFS4ERR_NOMATCHING_LAYOUT)
+#define nfserr_recallconflict		cpu_to_be32(NFS4ERR_RECALLCONFLICT)
+#define nfserr_unknown_layouttype	cpu_to_be32(NFS4ERR_UNKNOWN_LAYOUTTYPE)
+#define nfserr_seq_misordered		cpu_to_be32(NFS4ERR_SEQ_MISORDERED)
+#define nfserr_sequence_pos		cpu_to_be32(NFS4ERR_SEQUENCE_POS)
+#define nfserr_req_too_big		cpu_to_be32(NFS4ERR_REQ_TOO_BIG)
+#define nfserr_rep_too_big		cpu_to_be32(NFS4ERR_REP_TOO_BIG)
+#define nfserr_rep_too_big_to_cache	cpu_to_be32(NFS4ERR_REP_TOO_BIG_TO_CACHE)
+#define nfserr_retry_uncached_rep	cpu_to_be32(NFS4ERR_RETRY_UNCACHED_REP)
+#define nfserr_unsafe_compound		cpu_to_be32(NFS4ERR_UNSAFE_COMPOUND)
+#define nfserr_too_many_ops		cpu_to_be32(NFS4ERR_TOO_MANY_OPS)
+#define nfserr_op_not_in_session	cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION)
+#define nfserr_hash_alg_unsupp		cpu_to_be32(NFS4ERR_HASH_ALG_UNSUPP)
+#define nfserr_clientid_busy		cpu_to_be32(NFS4ERR_CLIENTID_BUSY)
+#define nfserr_pnfs_io_hole		cpu_to_be32(NFS4ERR_PNFS_IO_HOLE)
+#define nfserr_seq_false_retry		cpu_to_be32(NFS4ERR_SEQ_FALSE_RETRY)
+#define nfserr_bad_high_slot		cpu_to_be32(NFS4ERR_BAD_HIGH_SLOT)
+#define nfserr_deadsession		cpu_to_be32(NFS4ERR_DEADSESSION)
+#define nfserr_encr_alg_unsupp		cpu_to_be32(NFS4ERR_ENCR_ALG_UNSUPP)
+#define nfserr_pnfs_no_layout		cpu_to_be32(NFS4ERR_PNFS_NO_LAYOUT)
+#define nfserr_not_only_op		cpu_to_be32(NFS4ERR_NOT_ONLY_OP)
+#define nfserr_wrong_cred		cpu_to_be32(NFS4ERR_WRONG_CRED)
+#define nfserr_wrong_type		cpu_to_be32(NFS4ERR_WRONG_TYPE)
+#define nfserr_dirdeleg_unavail		cpu_to_be32(NFS4ERR_DIRDELEG_UNAVAIL)
+#define nfserr_reject_deleg		cpu_to_be32(NFS4ERR_REJECT_DELEG)
+#define nfserr_returnconflict		cpu_to_be32(NFS4ERR_RETURNCONFLICT)
+#define nfserr_deleg_revoked		cpu_to_be32(NFS4ERR_DELEG_REVOKED)
 
 /* error codes for internal use */
 /* if a request fails due to kmalloc failure, it gets dropped.
@@ -258,6 +295,10 @@ void		nfsd_lockd_shutdown(void);
 #define	nfserr_dropit		cpu_to_be32(30000)
 /* end-of-file indicator in readdir */
 #define	nfserr_eof		cpu_to_be32(30001)
+/* replay detected */
+#define	nfserr_replay_me	cpu_to_be32(11001)
+/* nfs41 replay detected */
+#define	nfserr_replay_cache	cpu_to_be32(11002)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)	(l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
-- 
1.6.2.1

[PATCH 09/47] nfsd41: sessions basic data types

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

This patch provides basic data structures representing the nfs41
sessions and slots, plus helpers for keeping a reference count
on the session and freeing it.

Note that our server only support a headerpadsz of 0 and
it ignores backchannel attributes at the moment.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: remove headerpadsz from channel attributes]
[nfsd41: embed nfsd4_channel in nfsd4_session]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |   22 ++++++++++++++++++++++
 include/linux/nfsd/state.h |   34 ++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 070e9e5..f4b624b 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -382,6 +382,28 @@ static void release_openowner(struct nfs4_stateowner *sop)
 	nfs4_put_stateowner(sop);
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+static void
+release_session(struct nfsd4_session *ses)
+{
+	list_del(&ses->se_hash);
+	list_del(&ses->se_perclnt);
+	nfsd4_put_session(ses);
+}
+
+void
+free_session(struct kref *kref)
+{
+	struct nfsd4_session *ses;
+
+	ses = container_of(kref, struct nfsd4_session, se_ref);
+	kfree(ses->se_slots);
+	kfree(ses);
+}
+#else /* CONFIG_NFSD_V4_1 */
+static inline void release_session(struct nfsd4_session *ses) {}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static inline void
 renew_client(struct nfs4_client *clp)
 {
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index a6e4a00..afae103 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -99,6 +99,40 @@ struct nfs4_callback {
 	struct rpc_clnt *       cb_client;
 };
 
+struct nfsd4_slot {
+	bool				sl_inuse;
+	struct nfsd4_session		*sl_session;
+	u32				sl_seqid;
+};
+
+struct nfsd4_session {
+	struct kref		se_ref;
+	struct list_head	se_hash;	/* hash by sessionid */
+	struct list_head	se_perclnt;
+	u32			se_flags;
+	struct nfs4_client	*se_client;	/* for expire_client */
+	struct nfs4_sessionid	se_sessionid;
+	u32			se_fmaxreq_sz;
+	u32			se_fmaxresp_sz;
+	u32			se_fmaxresp_cached;
+	u32			se_fmaxops;
+	u32			se_fnumslots;
+	struct nfsd4_slot	*se_slots;	/* forward channel slots */
+};
+
+static inline void
+nfsd4_put_session(struct nfsd4_session *ses)
+{
+	extern void free_session(struct kref *kref);
+	kref_put(&ses->se_ref, free_session);
+}
+
+static inline void
+nfsd4_get_session(struct nfsd4_session *ses)
+{
+	kref_get(&ses->se_ref);
+}
+
 #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
 
 /*
-- 
1.6.2.1

[PATCH 10/47] nfsd41: introduce nfs4_client cl_sessions list

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |    3 +++
 include/linux/nfsd/state.h |    3 +++
 2 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f4b624b..b71ec72 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -532,6 +532,9 @@ static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
 	INIT_LIST_HEAD(&clp->cl_strhash);
 	INIT_LIST_HEAD(&clp->cl_openowners);
 	INIT_LIST_HEAD(&clp->cl_delegations);
+#if defined(CONFIG_NFSD_V4_1)
+	INIT_LIST_HEAD(&clp->cl_sessions);
+#endif /* CONFIG_NFSD_V4_1 */
 	INIT_LIST_HEAD(&clp->cl_lru);
 	return clp;
 }
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index afae103..29624b4 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -164,6 +164,9 @@ struct nfs4_client {
 	struct nfs4_callback	cl_callback;    /* callback info */
 	atomic_t		cl_count;	/* ref count */
 	u32			cl_firststate;	/* recovery dir creation */
+#ifdef CONFIG_NFSD_V4_1
+	struct list_head	cl_sessions;
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 /* struct nfs4_client_reset
-- 
1.6.2.1

[PATCH 11/47] nfsd41: release_session when client is expired

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[add CONFIG_NFSD_V4_1 to fix v4.0 regression bug]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index b71ec72..df9d42e 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -515,6 +515,14 @@ expire_client(struct nfs4_client *clp)
 		sop = list_entry(clp->cl_openowners.next, struct nfs4_stateowner, so_perclient);
 		release_openowner(sop);
 	}
+#ifdef CONFIG_NFSD_V4_1
+	while (!list_empty(&clp->cl_sessions)) {
+		struct nfsd4_session  *ses;
+		ses = list_entry(clp->cl_sessions.next, struct nfsd4_session,
+				 se_perclnt);
+		release_session(ses);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 	put_nfs4_client(clp);
 }
 
-- 
1.6.2.1

[PATCH 12/47] nfsd41: sessionid hashing

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Simple sessionid hashing using its monotonically increasing sequence number.

Locking considerations:
sessionid_hashtbl access is controlled by the sessionid_lock spin lock.
It must be taken for insert, delete, and lookup.
nfsd4_sequence looks up the session id and if the session is found,
it calls nfsd4_get_session (still under the sessionid_lock).
nfsd4_destroy_session calls nfsd4_put_session after unhashing
it, so when the session's kref reaches zero it's going to get freed.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[we don't use a prime for sessionid hash table size]
[use sessionid_lock spin lock]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |   57 +++++++++++++++++++++++++++++++++++++++++++-
 include/linux/nfsd/state.h |    7 +++++
 2 files changed, 63 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index df9d42e..ac4e8f2 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -383,11 +383,62 @@ static void release_openowner(struct nfs4_stateowner *sop)
 }
 
 #if defined(CONFIG_NFSD_V4_1)
+static DEFINE_SPINLOCK(sessionid_lock);
+#define SESSION_HASH_SIZE	512
+static struct list_head sessionid_hashtbl[SESSION_HASH_SIZE];
+
+static inline int
+hash_sessionid(struct nfs4_sessionid *sessionid)
+{
+	struct nfsd4_sessionid *sid = (struct nfsd4_sessionid *)sessionid;
+
+	return sid->sequence % SESSION_HASH_SIZE;
+}
+
+static inline void
+dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
+{
+	u32 *ptr = (u32 *)(&sessionid->data[0]);
+	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
+}
+
+/* caller must hold sessionid_lock */
+static struct nfsd4_session *
+find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
+{
+	struct nfsd4_session *elem;
+	int idx;
+
+	dump_sessionid(__func__, sessionid);
+	idx = hash_sessionid(sessionid);
+	dprintk("%s: idx is %d\n", __func__, idx);
+	/* Search in the appropriate list */
+	list_for_each_entry(elem, &sessionid_hashtbl[idx], se_hash) {
+		dump_sessionid("list traversal", &elem->se_sessionid);
+		if (!memcmp(elem->se_sessionid.data, sessionid->data,
+			    NFS4_MAX_SESSIONID_LEN)) {
+			return elem;
+		}
+	}
+
+	dprintk("%s: session not found\n", __func__);
+	return NULL;
+}
+
+/* caller must hold sessionid_lock */
 static void
-release_session(struct nfsd4_session *ses)
+unhash_session(struct nfsd4_session *ses)
 {
 	list_del(&ses->se_hash);
 	list_del(&ses->se_perclnt);
+}
+
+static void
+release_session(struct nfsd4_session *ses)
+{
+	spin_lock(&sessionid_lock);
+	unhash_session(ses);
+	spin_unlock(&sessionid_lock);
 	nfsd4_put_session(ses);
 }
 
@@ -3213,6 +3264,10 @@ nfs4_state_init(void)
 		INIT_LIST_HEAD(&unconf_str_hashtbl[i]);
 		INIT_LIST_HEAD(&unconf_id_hashtbl[i]);
 	}
+#if defined(CONFIG_NFSD_V4_1)
+	for (i = 0; i < SESSION_HASH_SIZE; i++)
+		INIT_LIST_HEAD(&sessionid_hashtbl[i]);
+#endif /* CONFIG_NFSD_V4_1 */
 	for (i = 0; i < FILE_HASH_SIZE; i++) {
 		INIT_LIST_HEAD(&file_hashtbl[i]);
 	}
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 29624b4..7592d7b 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -133,6 +133,13 @@ nfsd4_get_session(struct nfsd4_session *ses)
 	kref_get(&ses->se_ref);
 }
 
+/* formatted contents of nfs4_sessionid */
+struct nfsd4_sessionid {
+	clientid_t	clientid;
+	u32		sequence;
+	u32		reserved;
+};
+
 #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
 
 /*
-- 
1.6.2.1

[PATCH 13/47] nfsd41: xdr infrastructure

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Define nfsd41_dec_ops vector and add it to nfsd4_minorversion for
minorversion 1.

Note: nfsd4_enc_ops vector is shared for v4.0 and v4.1
since we don't need to filter out obsolete ops as this is
done in the decoding phase.

exchange_id, create_session, destroy_session, and sequence ops are
implemented as stubs returning nfserr_opnotsupp at this stage.

[was nfsd41: xdr stubs]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c         |  156 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfsd/xdr4.h |   24 +++++++
 2 files changed, 180 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 76a0b2a..b082d07 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -996,6 +996,36 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
 	DECODE_TAIL;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+static __be32
+nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
+			 struct nfsd4_exchange_id *clid)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+
+static __be32
+nfsd4_decode_create_session(struct nfsd4_compoundargs *argp,
+			    struct nfsd4_create_session *sess)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+
+static __be32
+nfsd4_decode_destroy_session(struct nfsd4_compoundargs *argp,
+			     struct nfsd4_destroy_session *destroy_session)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+
+static __be32
+nfsd4_decode_sequence(struct nfsd4_compoundargs *argp,
+		      struct nfsd4_sequence *seq)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static __be32
 nfsd4_decode_noop(struct nfsd4_compoundargs *argp, void *p)
 {
@@ -1050,6 +1080,69 @@ static nfsd4_dec nfsd4_dec_ops[] = {
 	[OP_RELEASE_LOCKOWNER]	= (nfsd4_dec)nfsd4_decode_release_lockowner,
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+static nfsd4_dec nfsd41_dec_ops[] = {
+	[OP_ACCESS]		(nfsd4_dec)nfsd4_decode_access,
+	[OP_CLOSE]		(nfsd4_dec)nfsd4_decode_close,
+	[OP_COMMIT]		(nfsd4_dec)nfsd4_decode_commit,
+	[OP_CREATE]		(nfsd4_dec)nfsd4_decode_create,
+	[OP_DELEGPURGE]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_DELEGRETURN]	(nfsd4_dec)nfsd4_decode_delegreturn,
+	[OP_GETATTR]		(nfsd4_dec)nfsd4_decode_getattr,
+	[OP_GETFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_LINK]		(nfsd4_dec)nfsd4_decode_link,
+	[OP_LOCK]		(nfsd4_dec)nfsd4_decode_lock,
+	[OP_LOCKT]		(nfsd4_dec)nfsd4_decode_lockt,
+	[OP_LOCKU]		(nfsd4_dec)nfsd4_decode_locku,
+	[OP_LOOKUP]		(nfsd4_dec)nfsd4_decode_lookup,
+	[OP_LOOKUPP]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_NVERIFY]		(nfsd4_dec)nfsd4_decode_verify,
+	[OP_OPEN]		(nfsd4_dec)nfsd4_decode_open,
+	[OP_OPENATTR]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OPEN_CONFIRM]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OPEN_DOWNGRADE]	(nfsd4_dec)nfsd4_decode_open_downgrade,
+	[OP_PUTFH]		(nfsd4_dec)nfsd4_decode_putfh,
+	[OP_PUTPUBFH]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_PUTROOTFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_READ]		(nfsd4_dec)nfsd4_decode_read,
+	[OP_READDIR]		(nfsd4_dec)nfsd4_decode_readdir,
+	[OP_READLINK]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_REMOVE]		(nfsd4_dec)nfsd4_decode_remove,
+	[OP_RENAME]		(nfsd4_dec)nfsd4_decode_rename,
+	[OP_RENEW]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_RESTOREFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_SAVEFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_SECINFO]		(nfsd4_dec)nfsd4_decode_secinfo,
+	[OP_SETATTR]		(nfsd4_dec)nfsd4_decode_setattr,
+	[OP_SETCLIENTID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_SETCLIENTID_CONFIRM](nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_VERIFY]		(nfsd4_dec)nfsd4_decode_verify,
+	[OP_WRITE]		(nfsd4_dec)nfsd4_decode_write,
+	[OP_RELEASE_LOCKOWNER]	(nfsd4_dec)nfsd4_decode_notsupp,
+
+	/* new operations for NFSv4.1 */
+	[OP_BACKCHANNEL_CTL]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_BIND_CONN_TO_SESSION](nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_EXCHANGE_ID]	(nfsd4_dec)nfsd4_decode_exchange_id,
+	[OP_CREATE_SESSION]	(nfsd4_dec)nfsd4_decode_create_session,
+	[OP_DESTROY_SESSION]	(nfsd4_dec)nfsd4_decode_destroy_session,
+	[OP_FREE_STATEID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_GET_DIR_DELEGATION]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_GETDEVICEINFO]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_GETDEVICELIST]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_LAYOUTCOMMIT]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_LAYOUTGET]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_LAYOUTRETURN]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_SECINFO_NO_NAME]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_SEQUENCE]		(nfsd4_dec)nfsd4_decode_sequence,
+	[OP_SET_SSV]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_TEST_STATEID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_WANT_DELEGATION]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_DESTROY_CLIENTID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_RECLAIM_COMPLETE]	(nfsd4_dec)nfsd4_decode_notsupp,
+};
+#endif /* CONFIG_NFSD_V4_1 */
+
 struct nfsd4_minorversion_ops {
 	nfsd4_dec *decoders;
 	int nops;
@@ -1057,6 +1150,9 @@ struct nfsd4_minorversion_ops {
 
 static struct nfsd4_minorversion_ops nfsd4_minorversion[] = {
 	[0] = { nfsd4_dec_ops, ARRAY_SIZE(nfsd4_dec_ops) },
+#if defined(CONFIG_NFSD_V4_1)
+	[1] = { nfsd41_dec_ops, ARRAY_SIZE(nfsd41_dec_ops) },
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 static __be32
@@ -2571,6 +2667,40 @@ nfsd4_encode_write(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_w
 	return nfserr;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+static __be32
+nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
+			 struct nfsd4_exchange_id *exid)
+{
+	/* stub */
+	return nfserr;
+}
+
+static __be32
+nfsd4_encode_create_session(struct nfsd4_compoundres *resp, int nfserr,
+			    struct nfsd4_create_session *sess)
+{
+	/* stub */
+	return nfserr;
+}
+
+static __be32
+nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
+			     struct nfsd4_destroy_session *destroy_session)
+{
+	/* stub */
+	return nfserr;
+}
+
+static __be32
+nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
+		      struct nfsd4_sequence *seq)
+{
+	/* stub */
+	return nfserr;
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static __be32
 nfsd4_encode_noop(struct nfsd4_compoundres *resp, __be32 nfserr, void *p)
 {
@@ -2579,6 +2709,11 @@ nfsd4_encode_noop(struct nfsd4_compoundres *resp, __be32 nfserr, void *p)
 
 typedef __be32(* nfsd4_enc)(struct nfsd4_compoundres *, __be32, void *);
 
+/*
+ * Note: nfsd4_enc_ops vector is shared for v4.0 and v4.1
+ * since we don't need to filter out obsolete ops as this is
+ * done in the decoding phase.
+ */
 static nfsd4_enc nfsd4_enc_ops[] = {
 	[OP_ACCESS]		= (nfsd4_enc)nfsd4_encode_access,
 	[OP_CLOSE]		= (nfsd4_enc)nfsd4_encode_close,
@@ -2617,6 +2752,27 @@ static nfsd4_enc nfsd4_enc_ops[] = {
 	[OP_VERIFY]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_WRITE]		= (nfsd4_enc)nfsd4_encode_write,
 	[OP_RELEASE_LOCKOWNER]	= (nfsd4_enc)nfsd4_encode_noop,
+#if defined(CONFIG_NFSD_V4_1)
+	[OP_BACKCHANNEL_CTL]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_BIND_CONN_TO_SESSION] = (nfsd4_enc)nfsd4_encode_noop,
+	[OP_EXCHANGE_ID]	= (nfsd4_enc)nfsd4_encode_exchange_id,
+	[OP_CREATE_SESSION]	= (nfsd4_enc)nfsd4_encode_create_session,
+	[OP_DESTROY_SESSION]	= (nfsd4_enc)nfsd4_encode_destroy_session,
+	[OP_FREE_STATEID]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_GET_DIR_DELEGATION]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_GETDEVICEINFO]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_GETDEVICELIST]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_LAYOUTCOMMIT]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_LAYOUTGET]		= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_LAYOUTRETURN]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_SECINFO_NO_NAME]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_SEQUENCE]		= (nfsd4_enc)nfsd4_encode_sequence,
+	[OP_SET_SSV]		= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_TEST_STATEID]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_WANT_DELEGATION]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_DESTROY_CLIENTID]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_RECLAIM_COMPLETE]	= (nfsd4_enc)nfsd4_encode_noop,
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 void
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index e309d55..40b148e 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -346,6 +346,24 @@ struct nfsd4_write {
 	nfs4_verifier	wr_verifier;        /* response */
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+struct nfsd4_exchange_id {
+	int	foo;	/* stub */
+};
+
+struct nfsd4_create_session {
+	int	foo;	/* stub */
+};
+
+struct nfsd4_sequence {
+	int	foo;	/* stub */
+};
+
+struct nfsd4_destroy_session {
+	int	foo;	/* stub */
+};
+#endif /* CONFIG_NFSD_V4_1 */
+
 struct nfsd4_op {
 	int					opnum;
 	__be32					status;
@@ -380,6 +398,12 @@ struct nfsd4_op {
 		struct nfsd4_verify		verify;
 		struct nfsd4_write		write;
 		struct nfsd4_release_lockowner	release_lockowner;
+#if defined(CONFIG_NFSD_V4_1)
+		struct nfsd4_exchange_id	exchange_id;
+		struct nfsd4_create_session	create_session;
+		struct nfsd4_destroy_session	destroy_session;
+		struct nfsd4_sequence		sequence;
+#endif /* CONFIG_NFSD_V4_1 */
 	} u;
 	struct nfs4_replay *			replay;
 };
-- 
1.6.2.1

[PATCH 14/47] nfsd: remove nfsd4_ops array size

[Benny Halevy]

There's no need for it.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index c8e1ad0..ded469f 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -951,7 +951,7 @@ out:
 	return status;
 }
 
-static struct nfsd4_operation nfsd4_ops[OP_RELEASE_LOCKOWNER+1] = {
+static struct nfsd4_operation nfsd4_ops[] = {
 	[OP_ACCESS] = {
 		.op_func = (nfsd4op_func)nfsd4_access,
 		.op_name = "OP_ACCESS",
-- 
1.6.2.1

[PATCH 15/47] nfsd41: proc stubs

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   22 ++++++++++++++++++++++
 fs/nfsd/nfs4state.c       |   34 ++++++++++++++++++++++++++++++++++
 include/linux/nfsd/xdr4.h |   14 ++++++++++++++
 3 files changed, 70 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index ded469f..f618e8e 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1101,6 +1101,28 @@ static struct nfsd4_operation nfsd4_ops[] = {
 		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_ON_ABSENT_FS,
 		.op_name = "OP_RELEASE_LOCKOWNER",
 	},
+#if defined(CONFIG_NFSD_V4_1)
+	[OP_EXCHANGE_ID] = {
+		.op_func = (nfsd4op_func)nfsd4_exchange_id,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_EXCHANGE_ID",
+	},
+	[OP_CREATE_SESSION] = {
+		.op_func = (nfsd4op_func)nfsd4_create_session,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_CREATE_SESSION",
+	},
+	[OP_DESTROY_SESSION] = {
+		.op_func = (nfsd4op_func)nfsd4_destroy_session,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_DESTROY_SESSION",
+	},
+	[OP_SEQUENCE] = {
+		.op_func = (nfsd4op_func)nfsd4_sequence,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_SEQUENCE",
+	},
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 static const char *nfsd4_op_name(unsigned opnum)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index ac4e8f2..bbb7455 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -840,6 +840,40 @@ out_err:
 	return;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+__be32
+nfsd4_exchange_id(struct svc_rqst *rqstp,
+		  struct nfsd4_compound_state *cstate,
+		  struct nfsd4_exchange_id *exid)
+{
+	return -1;	/* stub */
+}
+
+__be32
+nfsd4_create_session(struct svc_rqst *rqstp,
+		     struct nfsd4_compound_state *cstate,
+		     struct nfsd4_create_session *cr_ses)
+{
+	return -1;	/* stub */
+}
+
+__be32
+nfsd4_destroy_session(struct svc_rqst *r,
+		      struct nfsd4_compound_state *cstate,
+		      struct nfsd4_destroy_session *sessionid)
+{
+	return -1;	/* stub */
+}
+
+__be32
+nfsd4_sequence(struct svc_rqst *r,
+	       struct nfsd4_compound_state *cstate,
+	       struct nfsd4_sequence *seq)
+{
+	return -1;	/* stub */
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 __be32
 nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		  struct nfsd4_setclientid *setclid)
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 40b148e..0148d54 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -475,6 +475,20 @@ extern __be32 nfsd4_setclientid(struct svc_rqst *rqstp,
 extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 		struct nfsd4_setclientid_confirm *setclientid_confirm);
+#if defined(CONFIG_NFSD_V4_1)
+extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
+		struct nfsd4_compound_state *,
+struct nfsd4_exchange_id *);
+		extern __be32 nfsd4_create_session(struct svc_rqst *,
+		struct nfsd4_compound_state *,
+		struct nfsd4_create_session *);
+extern __be32 nfsd4_sequence(struct svc_rqst *,
+		struct nfsd4_compound_state *,
+		struct nfsd4_sequence *);
+extern __be32 nfsd4_destroy_session(struct svc_rqst *,
+		struct nfsd4_compound_state *,
+		struct nfsd4_destroy_session *);
+#endif /* CONFIG_NFSD_V4_1 */
 extern __be32 nfsd4_process_open1(struct nfsd4_open *open);
 extern __be32 nfsd4_process_open2(struct svc_rqst *rqstp,
 		struct svc_fh *current_fh, struct nfsd4_open *open);
-- 
1.6.2.1

[PATCH 16/47] nfsd41: exchange_id operation

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the exchange_id operation confoming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-28

Based on the client provided name, hash a client id.
If a confirmed one is found, compare the op's creds and
verifier.  If the creds match and the verifier is different
then expire the old client (client re-incarnated), otherwise,
if both match, assume it's a replay and ignore it.

If an unconfirmed client is found, then copy the new creds
and verifer if need update, otherwise assume replay.

The client is moved to a confirmed state on create_session.

In the nfs41 branch set the exchange_id flags to
EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_SUPP_MOVED_REFER
(pNFS is not supported, Referrals are supported,
Migration is not.).

Address various scenarios from section 18.35 of the spec:

1. Check for EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and set
   EXCHGID4_FLAG_CONFIRMED_R as appropriate.

2. Return error codes per 18.35.4 scenarios.

3. Update client records or generate new client ids depending on
   scenario.

Note: 18.35.4 case 3 probably still needs revisiting.  The handling
seems not quite right.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use utsname for major_id (and copy to server_scope)]
[nfsd41: fix handling of various exchange id scenarios]
Signed-off-by: Mike Sager <sager@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |  138 +++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c          |  146 +++++++++++++++++++++++++++++++++++++++++++-
 include/linux/nfsd/state.h |    2 +
 include/linux/nfsd/xdr4.h  |    8 ++-
 4 files changed, 289 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index bbb7455..09c63ff 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -841,12 +841,148 @@ out_err:
 }
 
 #if defined(CONFIG_NFSD_V4_1)
+/*
+ * Set the exchange_id flags returned by the server.
+ */
+static void
+nfsd4_set_ex_flags(struct nfs4_client *new, struct nfsd4_exchange_id *clid)
+{
+	/* pNFS is not supported */
+	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_NON_PNFS;
+
+	/* Referrals are supported, Migration is not. */
+	new->cl_exchange_flags |= EXCHGID4_FLAG_SUPP_MOVED_REFER;
+
+	/* set the wire flags to return to client. */
+	clid->flags = new->cl_exchange_flags;
+}
+
 __be32
 nfsd4_exchange_id(struct svc_rqst *rqstp,
 		  struct nfsd4_compound_state *cstate,
 		  struct nfsd4_exchange_id *exid)
 {
-	return -1;	/* stub */
+	struct nfs4_client *unconf, *conf, *new;
+	int status;
+	unsigned int		strhashval;
+	char			dname[HEXDIR_LEN];
+	nfs4_verifier		verf = exid->verifier;
+	u32			ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
+	struct xdr_netobj clname = {
+		.len = exid->id_len,
+		.data = exid->id,
+	};
+
+	dprintk("%s rqstp=%p exid=%p clname.len=%u clname.data=%p "
+		" ip_addr=%u flags %x, spa_how %d\n",
+		__func__, rqstp, exid, clname.len, clname.data,
+		ip_addr, exid->flags, exid->spa_how);
+
+	if (!check_name(clname) || (exid->flags & EXCHGID4_INVAL_FLAG_MASK_A))
+		return nfserr_inval;
+
+	/* Currently only support SP4_NONE */
+	if (exid->spa_how != SP4_NONE)
+		return nfserr_encr_alg_unsupp;
+
+	status = nfs4_make_rec_clidname(dname, &clname);
+
+	if (status)
+		goto error;
+
+	strhashval = clientstr_hashval(dname);
+
+	nfs4_lock_state();
+	status = nfs_ok;
+
+	conf = find_confirmed_client_by_str(dname, strhashval);
+	if (conf) {
+		if (!same_verf(&verf, &conf->cl_verifier)) {
+			/* 18.35.4 case 8 */
+			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
+				status = nfserr_not_same;
+				goto out;
+			}
+			/* Client reboot: destroy old state */
+			expire_client(conf);
+			goto out_new;
+		}
+		if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) {
+			/* 18.35.4 case 9 */
+			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
+				status = nfserr_perm;
+				goto out;
+			}
+			expire_client(conf);
+			goto out_new;
+		}
+		if (ip_addr != conf->cl_addr &&
+		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A)) {
+			/* Client collision. 18.35.4 case 3 */
+			status = nfserr_clid_inuse;
+			goto out;
+		}
+		/*
+		 * Set bit when the owner id and verifier map to an already
+		 * confirmed client id (18.35.3).
+		 */
+		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
+
+		/*
+		 * Falling into 18.35.4 case 2, possible router replay.
+		 * Leave confirmed record intact and return same result.
+		 */
+		copy_verf(conf, &verf);
+		new = conf;
+		goto out_copy;
+	} else {
+		/* 18.35.4 case 7 */
+		if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
+			status = nfserr_noent;
+			goto out;
+		}
+	}
+
+	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
+	if (unconf) {
+		/*
+		 * Possible retry or client restart.  Per 18.35.4 case 4,
+		 * a new unconfirmed record should be generated regardless
+		 * of whether any properties have changed.
+		 */
+		expire_client(unconf);
+	}
+
+out_new:
+	/* Normal case */
+	new = create_client(clname, dname);
+	if (new == NULL) {
+		status = nfserr_resource;
+		goto out;
+	}
+
+	copy_verf(new, &verf);
+	copy_cred(&new->cl_cred, &rqstp->rq_cred);
+	new->cl_addr = ip_addr;
+	gen_clid(new);
+	gen_confirm(new);
+	add_to_unconfirmed(new, strhashval);
+out_copy:
+	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
+	exid->clientid.cl_id = new->cl_clientid.cl_id;
+
+	new->cl_seqid = exid->seqid = 1;
+	nfsd4_set_ex_flags(new, exid);
+
+	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
+		new->cl_seqid, new->cl_exchange_flags);
+	status = nfs_ok;
+
+out:
+	nfs4_unlock_state();
+error:
+	dprintk("nfsd4_exchange_id returns %d\n", ntohl(status));
+	return status;
 }
 
 __be32
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index b082d07..840cf6a 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -45,6 +45,7 @@
 #include <linux/fs.h>
 #include <linux/namei.h>
 #include <linux/vfs.h>
+#include <linux/utsname.h>
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
@@ -999,9 +1000,100 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
 #if defined(CONFIG_NFSD_V4_1)
 static __be32
 nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
-			 struct nfsd4_exchange_id *clid)
+			 struct nfsd4_exchange_id *exid)
 {
-	return nfserr_opnotsupp;	/* stub */
+	int dummy;
+	DECODE_HEAD;
+
+	READ_BUF(NFS4_VERIFIER_SIZE);
+	COPYMEM(exid->verifier.data, NFS4_VERIFIER_SIZE);
+
+	READ_BUF(4);
+	READ32(exid->id_len);
+
+	READ_BUF(exid->id_len);
+	SAVEMEM(exid->id, exid->id_len);
+
+	READ_BUF(4);
+	READ32(exid->flags);
+
+	/* Ignore state_protect4_a */
+	READ_BUF(4);
+	READ32(exid->spa_how);
+	switch (exid->spa_how) {
+	case SP4_NONE:
+		break;
+	case SP4_MACH_CRED:
+		/* spo_must_enforce */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+
+		/* spo_must_allow */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+		break;
+	case SP4_SSV:
+		/* ssp_ops */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+
+		/* ssp_hash_algs<> */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* ssp_encr_algs<> */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* ssp_window and ssp_num_gss_handles */
+		READ_BUF(8);
+		READ32(dummy);
+		READ32(dummy);
+		break;
+	default:
+		goto xdr_error;
+	}
+
+	/* Ignore Implementation ID */
+	READ_BUF(4);    /* nfs_impl_id4 array length */
+	READ32(dummy);
+
+	if (dummy > 1)
+		goto xdr_error;
+
+	if (dummy == 1) {
+		/* nii_domain */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* nii_name */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* nii_date */
+		READ_BUF(12);
+		p += 3;
+	}
+	DECODE_TAIL;
 }
 
 static __be32
@@ -2672,7 +2764,55 @@ static __be32
 nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
 			 struct nfsd4_exchange_id *exid)
 {
-	/* stub */
+	ENCODE_HEAD;
+	char *major_id;
+	char *server_scope;
+	int major_id_sz;
+	int server_scope_sz;
+	uint64_t minor_id = 0;
+
+	if (nfserr)
+		goto out;
+
+	major_id = utsname()->nodename;
+	major_id_sz = strlen(major_id);
+	server_scope = utsname()->nodename;
+	server_scope_sz = strlen(server_scope);
+
+	RESERVE_SPACE(
+		8 /* eir_clientid */ +
+		4 /* eir_sequenceid */ +
+		4 /* eir_flags */ +
+		4 /* spr_how (SP4_NONE) */ +
+		8 /* so_minor_id */ +
+		4 /* so_major_id.len */ +
+		(XDR_QUADLEN(major_id_sz) * 4) +
+		4 /* eir_server_scope.len */ +
+		(XDR_QUADLEN(server_scope_sz) * 4) +
+		4 /* eir_server_impl_id.count (0) */);
+
+	WRITEMEM(&exid->clientid, 8);
+	WRITE32(exid->seqid);
+	WRITE32(exid->flags);
+
+	/* state_protect4_r. Currently only support SP4_NONE */
+	BUG_ON(exid->spa_how != SP4_NONE);
+	WRITE32(exid->spa_how);
+
+	/* The server_owner struct */
+	WRITE64(minor_id);      /* Minor id */
+	/* major id */
+	WRITE32(major_id_sz);
+	WRITEMEM(major_id, major_id_sz);
+
+	/* Server scope */
+	WRITE32(server_scope_sz);
+	WRITEMEM(server_scope, server_scope_sz);
+
+	/* Implementation id */
+	WRITE32(0);	/* zero length nfs_impl_id4 array */
+	ADJUST_ARGS();
+out:
 	return nfserr;
 }
 
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 7592d7b..5de36a7 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -173,6 +173,8 @@ struct nfs4_client {
 	u32			cl_firststate;	/* recovery dir creation */
 #ifdef CONFIG_NFSD_V4_1
 	struct list_head	cl_sessions;
+	u32			cl_seqid;       /* seqid for create_session */
+	u32			cl_exchange_flags;
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 0148d54..ea5a427 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -348,7 +348,13 @@ struct nfsd4_write {
 
 #if defined(CONFIG_NFSD_V4_1)
 struct nfsd4_exchange_id {
-	int	foo;	/* stub */
+	nfs4_verifier	verifier;
+	u32		id_len;
+	char		*id;
+	u32		flags;
+	clientid_t	clientid;
+	u32		seqid;
+	int		spa_how;
 };
 
 struct nfsd4_create_session {
-- 
1.6.2.1

[PATCH 17/47] nfsd41: match clientid establishment method

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

We need to distinguish between client names provided by NFSv4.0 clients
SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
up the clientid by string.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
[nfsd41: use boolean values for use_exchange_id argument]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4recover.c      |    3 ++-
 fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
 include/linux/nfsd/state.h |    2 +-
 3 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index b11cf8d..3444c00 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
 {
 	int status;
 
-	if (nfs4_has_reclaimed_state(child->d_name.name))
+	/* note: we currently use this path only for minorversion 0 */
+	if (nfs4_has_reclaimed_state(child->d_name.name, false))
 		return 0;
 
 	status = nfsd4_clear_clid_dir(parent, child);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 09c63ff..0c39376 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
 	return NULL;
 }
 
+/*
+ * Return 1 iff clp's clientid establishment method matches the use_exchange_id
+ * parameter. Matching is based on the fact the at least one of the
+ * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
+ */
+static inline int
+match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
+{
+#if defined(CONFIG_NFSD_V4_1)
+	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
+#else  /* CONFIG_NFSD_V4_1 */
+	return 1;
+#endif /* CONFIG_NFSD_V4_1 */
+}
+
 static struct nfs4_client *
-find_confirmed_client_by_str(const char *dname, unsigned int hashval)
+find_confirmed_client_by_str(const char *dname, unsigned int hashval,
+			     bool use_exchange_id)
 {
 	struct nfs4_client *clp;
 
 	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
-		if (same_name(clp->cl_recdir, dname))
+		if (same_name(clp->cl_recdir, dname) &&
+		    match_clientid_establishment(clp, use_exchange_id))
 			return clp;
 	}
 	return NULL;
 }
 
 static struct nfs4_client *
-find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
+find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
+			       bool use_exchange_id)
 {
 	struct nfs4_client *clp;
 
 	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
-		if (same_name(clp->cl_recdir, dname))
+		if (same_name(clp->cl_recdir, dname) &&
+		    match_clientid_establishment(clp, use_exchange_id))
 			return clp;
 	}
 	return NULL;
@@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
 	nfs4_lock_state();
 	status = nfs_ok;
 
-	conf = find_confirmed_client_by_str(dname, strhashval);
+	conf = find_confirmed_client_by_str(dname, strhashval, true);
 	if (conf) {
 		if (!same_verf(&verf, &conf->cl_verifier)) {
 			/* 18.35.4 case 8 */
@@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
 		}
 	}
 
-	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
+	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
 	if (unconf) {
 		/*
 		 * Possible retry or client restart.  Per 18.35.4 case 4,
@@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	strhashval = clientstr_hashval(dname);
 
 	nfs4_lock_state();
-	conf = find_confirmed_client_by_str(dname, strhashval);
+	conf = find_confirmed_client_by_str(dname, strhashval, false);
 	if (conf) {
 		/* RFC 3530 14.2.33 CASE 0: */
 		status = nfserr_clid_inuse;
@@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	 * has a description of SETCLIENTID request processing consisting
 	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
 	 */
-	unconf = find_unconfirmed_client_by_str(dname, strhashval);
+	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
 	status = nfserr_resource;
 	if (!conf) {
 		/*
@@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 			unsigned int hash =
 				clientstr_hashval(unconf->cl_recdir);
 			conf = find_confirmed_client_by_str(unconf->cl_recdir,
-									hash);
+							    hash, false);
 			if (conf) {
 				nfsd4_remove_clid_dir(conf);
 				expire_client(conf);
@@ -3332,12 +3351,12 @@ alloc_reclaim(void)
 }
 
 int
-nfs4_has_reclaimed_state(const char *name)
+nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
 {
 	unsigned int strhashval = clientstr_hashval(name);
 	struct nfs4_client *clp;
 
-	clp = find_confirmed_client_by_str(name, strhashval);
+	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
 	return clp ? 1 : 0;
 }
 
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 5de36a7..feab6ec 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
 extern int nfsd4_recdir_load(void);
 extern void nfsd4_shutdown_recdir(void);
 extern int nfs4_client_to_reclaim(const char *name);
-extern int nfs4_has_reclaimed_state(const char *name);
+extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
 extern void nfsd4_recdir_purge_old(void);
 extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
 extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
-- 
1.6.2.1

[PATCH 18/47] nfsd41: sequence operation

[Benny Halevy]

Implement the sequence operation conforming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

Check for stale clientid (as derived from the sessionid).
Enforce slotid range and exactly-once semantics using
the slotid and seqid.

If everything went well renew the client lease and
mark the slot INPROGRESS.

[nfsd41: rename sequence catchthis to cachethis]
Signed-off-by: Andy Adamson<andros@netapp.com>
[pulled some code to set cstate->slot from "nfsd DRC logic"]
[use sessionid_lock spin lock]
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c       |   71 +++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c         |   32 +++++++++++++++++++-
 include/linux/nfsd/xdr4.h |   10 ++++++-
 3 files changed, 108 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 0c39376..a19f292 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1004,6 +1004,32 @@ error:
 	return status;
 }
 
+static int
+check_slot_seqid(u32 seqid, struct nfsd4_slot *slot)
+{
+	dprintk("%s enter. seqid %d slot->sl_seqid %d\n", __func__, seqid,
+		slot->sl_seqid);
+
+	/* The slot is in use, and no response has been sent. */
+	if (slot->sl_inuse) {
+		if (seqid == slot->sl_seqid)
+			return nfserr_jukebox;
+		else
+			return nfserr_seq_misordered;
+	}
+	/* Normal */
+	if (likely(seqid == slot->sl_seqid + 1))
+		return nfs_ok;
+	/* Replay */
+	if (seqid == slot->sl_seqid)
+		return nfserr_replay_cache;
+	/* Wraparound */
+	if (seqid == 1 && (slot->sl_seqid + 1) == 0)
+		return nfs_ok;
+	/* Misordered replay or misordered new request */
+	return nfserr_seq_misordered;
+}
+
 __be32
 nfsd4_create_session(struct svc_rqst *rqstp,
 		     struct nfsd4_compound_state *cstate,
@@ -1021,11 +1047,52 @@ nfsd4_destroy_session(struct svc_rqst *r,
 }
 
 __be32
-nfsd4_sequence(struct svc_rqst *r,
+nfsd4_sequence(struct svc_rqst *rqstp,
 	       struct nfsd4_compound_state *cstate,
 	       struct nfsd4_sequence *seq)
 {
-	return -1;	/* stub */
+	struct nfsd4_session *session;
+	struct nfsd4_slot *slot;
+	int status;
+
+	spin_lock(&sessionid_lock);
+	status = nfserr_badsession;
+	session = find_in_sessionid_hashtbl(&seq->sessionid);
+	if (!session)
+		goto out;
+
+	status = nfserr_badslot;
+	if (seq->slotid >= session->se_fnumslots)
+		goto out;
+
+	slot = &session->se_slots[seq->slotid];
+	dprintk("%s: slotid %d\n", __func__, seq->slotid);
+
+	status = check_slot_seqid(seq->seqid, slot);
+	if (status == nfserr_replay_cache) {
+		cstate->slot = slot;
+		goto replay_cache;
+	}
+	if (status)
+		goto out;
+
+	/* Success! bump slot seqid */
+	slot->sl_inuse = true;
+	slot->sl_seqid = seq->seqid;
+
+	cstate->slot = slot;
+
+replay_cache:
+	/* Renew the clientid on success and on replay.
+	 * Hold a session reference until done processing the compound:
+	 * nfsd4_put_session called only if the cstate slot is set.
+	 */
+	renew_client(session->se_client);
+	nfsd4_get_session(slot->sl_session);
+out:
+	spin_unlock(&sessionid_lock);
+	dprintk("%s: return %d\n", __func__, ntohl(status));
+	return status;
 }
 #endif /* CONFIG_NFSD_V4_1 */
 
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 840cf6a..c6b490e 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1114,7 +1114,16 @@ static __be32
 nfsd4_decode_sequence(struct nfsd4_compoundargs *argp,
 		      struct nfsd4_sequence *seq)
 {
-	return nfserr_opnotsupp;	/* stub */
+	DECODE_HEAD;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	COPYMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	READ32(seq->seqid);
+	READ32(seq->slotid);
+	READ32(seq->maxslots);
+	READ32(seq->cachethis);
+
+	DECODE_TAIL;
 }
 #endif /* CONFIG_NFSD_V4_1 */
 
@@ -2836,7 +2845,26 @@ static __be32
 nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
 		      struct nfsd4_sequence *seq)
 {
-	/* stub */
+	ENCODE_HEAD;
+
+	if (nfserr)
+		goto out;
+
+	RESERVE_SPACE(NFS4_MAX_SESSIONID_LEN + 20);
+	WRITEMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(seq->seqid);
+	WRITE32(seq->slotid);
+	WRITE32(seq->maxslots);
+	/*
+	 * FIXME: for now:
+	 *   target_maxslots = maxslots
+	 *   status_flags = 0
+	 */
+	WRITE32(seq->maxslots);
+	WRITE32(0);
+
+	ADJUST_ARGS();
+out:
 	return nfserr;
 }
 #endif /* CONFIG_NFSD_V4_1 */
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index ea5a427..9e4d8db 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -362,7 +362,15 @@ struct nfsd4_create_session {
 };
 
 struct nfsd4_sequence {
-	int	foo;	/* stub */
+	struct nfs4_sessionid	sessionid;		/* request/response */
+	u32			seqid;			/* request/response */
+	u32			slotid;			/* request/response */
+	u32			maxslots;		/* request/response */
+	u32			cachethis;		/* request */
+#if 0
+	u32			target_maxslots;	/* response */
+	u32			status_flags;		/* response */
+#endif /* not yet */
 };
 
 struct nfsd4_destroy_session {
-- 
1.6.2.1

[PATCH 19/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c  |   24 ++++++++++++++++--------
 fs/nfsd/nfs4state.c |    4 ++++
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f618e8e..e703ac2 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -811,14 +811,15 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
 
 typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
 			      void *);
+enum nfsd4_op_flags {
+	ALLOWED_WITHOUT_FH = 1 << 0,	/* No current filehandle required */
+	ALLOWED_ON_ABSENT_FS = 2 << 0,	/* ops processed on absent fs */
+	ALLOWED_AS_FIRST_OP = 3 << 0,	/* ops reqired first in compound */
+};
 
 struct nfsd4_operation {
 	nfsd4op_func op_func;
 	u32 op_flags;
-/* Most ops require a valid current filehandle; a few don't: */
-#define ALLOWED_WITHOUT_FH 1
-/* GETATTR and ops not listed as returning NFS4ERR_MOVED: */
-#define ALLOWED_ON_ABSENT_FS 2
 	char *op_name;
 };
 
@@ -864,6 +865,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
 		goto out;
 
+	op = &args->ops[0];
+	if (args->opcnt > 0 && op->status != nfserr_op_illegal &&
+	    !(nfsd4_ops[op->opnum].op_flags & ALLOWED_AS_FIRST_OP)) {
+		op->status = nfserr_sequence_pos;
+		goto encode_op;
+	}
+
 	status = nfs_ok;
 	while (!status && resp->opcnt < args->opcnt) {
 		op = &args->ops[resp->opcnt++];
@@ -1104,22 +1112,22 @@ static struct nfsd4_operation nfsd4_ops[] = {
 #if defined(CONFIG_NFSD_V4_1)
 	[OP_EXCHANGE_ID] = {
 		.op_func = (nfsd4op_func)nfsd4_exchange_id,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_EXCHANGE_ID",
 	},
 	[OP_CREATE_SESSION] = {
 		.op_func = (nfsd4op_func)nfsd4_create_session,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_CREATE_SESSION",
 	},
 	[OP_DESTROY_SESSION] = {
 		.op_func = (nfsd4op_func)nfsd4_destroy_session,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_DESTROY_SESSION",
 	},
 	[OP_SEQUENCE] = {
 		.op_func = (nfsd4op_func)nfsd4_sequence,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_SEQUENCE",
 	},
 #endif /* CONFIG_NFSD_V4_1 */
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index a19f292..10eb67b 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1051,10 +1051,14 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	       struct nfsd4_compound_state *cstate,
 	       struct nfsd4_sequence *seq)
 {
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfsd4_session *session;
 	struct nfsd4_slot *slot;
 	int status;
 
+	if (resp->opcnt != 1)
+		return nfserr_sequence_pos;
+
 	spin_lock(&sessionid_lock);
 	status = nfserr_badsession;
 	session = find_in_sessionid_hashtbl(&seq->sessionid);
-- 
1.6.2.1

[PATCH 20/47] nfsd41: DRC save, restore, and clear functions

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Cache all the result pages, including the rpc header in rq_respages[0],
for a request in the slot table cache entry.

Cache the statp pointer from nfsd_dispatch which points into rq_respages[0]
just past the rpc header. When setting a cache entry, calculate and save the
length of the nfs data minus the rpc header for rq_respages[0].

When replaying a cache entry, replace the cached rpc header with the
replayed request rpc result header, unless there is not enough room in the
cached results first page. In that case, use the cached rpc header.

The sessions fore channel maxresponse size cached is set to NFSD_PAGES_PER_SLOT
* PAGE_SIZE. For compounds we are cacheing with operations such as READDIR
that use the xdr_buf->pages to hold data, we choose to cache the extra page of
data rather than copying data from xdr_buf->pages into the xdr_buf->head page.

[nfsd41: limit cache to maxresponsesize_cached]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfssvc.c           |    4 +
 include/linux/nfsd/cache.h |    5 ++
 include/linux/nfsd/state.h |   13 ++++
 include/linux/nfsd/xdr4.h  |    4 +
 5 files changed, 168 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 10eb67b..f0ce639 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -860,6 +860,148 @@ out_err:
 }
 
 #if defined(CONFIG_NFSD_V4_1)
+void
+nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
+{
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
+
+	resp->cstate.statp = statp;
+}
+
+/*
+ * Dereference the result pages.
+ */
+static void
+nfsd4_release_respages(struct page **respages, short resused)
+{
+	int page_no;
+
+	dprintk("--> %s\n", __func__);
+	for (page_no = 0; page_no < resused; page_no++) {
+		if (!respages[page_no])
+			continue;
+		put_page(respages[page_no]);
+		respages[page_no] = NULL;
+	}
+}
+
+static void
+nfsd4_move_pages(struct page **topages, struct page **frompages, short count)
+{
+	int page_no;
+
+	for (page_no = 0; page_no < count; page_no++) {
+		topages[page_no] = frompages[page_no];
+		if (!topages[page_no])
+			continue;
+		get_page(topages[page_no]);
+	}
+}
+
+/*
+ * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing the previous
+ * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where the total
+ * length of the XDR response is less than se_fmaxresp_cached
+ * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used for a
+ * of the reply (e.g. readdir).
+ *
+ * Store the base and length of the rq_req.head[0] page
+ * of the NFSv4.1 data, just past the rpc header.
+ */
+void
+nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
+	struct svc_rqst *rqstp = resp->rqstp;
+	struct kvec *resv = &rqstp->rq_res.head[0];
+
+	dprintk("--> %s entry %p\n", __func__, entry);
+
+	/* Don't cache a failed OP_SEQUENCE */
+	if (resp->opcnt == 1 && resp->cstate.status)
+		return;
+	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
+	entry->ce_resused = rqstp->rq_resused;
+	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
+		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
+	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
+			 entry->ce_resused);
+	entry->ce_status = resp->cstate.status;
+	entry->ce_datav.iov_base = resp->cstate.statp;
+	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
+				(char *)page_address(rqstp->rq_respages[0]));
+	entry->ce_opcnt = resp->opcnt;
+	/* Current request rpc header length*/
+	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
+				(char *)page_address(rqstp->rq_respages[0]);
+}
+
+/*
+ * Copy the cached NFSv4.1 reply skipping the cached rpc header into the
+ * replay result res.head[0] past the rpc header to end up with replay
+ * rpc header and cached NFSv4.1 reply.
+ */
+static int
+nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
+			struct nfsd4_cache_entry *entry)
+{
+	struct svc_rqst *rqstp = resp->rqstp;
+	struct kvec *resv = &resp->rqstp->rq_res.head[0];
+	int len;
+
+	/* Current request rpc header length*/
+	len = (char *)resp->cstate.statp -
+			(char *)page_address(rqstp->rq_respages[0]);
+	if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
+		dprintk("%s v41 cached reply too large (%Zd).\n", __func__,
+			entry->ce_datav.iov_len);
+		return 0;
+	}
+	/* copy the cached reply nfsd data past the current rpc header */
+	memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_base,
+		entry->ce_datav.iov_len);
+	resv->iov_len = len + entry->ce_datav.iov_len;
+	return 1;
+}
+
+/*
+ * Keep the first page of the replay. Copy the NFSv4.1 data from the first
+ * cached page.  Replace any futher replay pages from the cache.
+ */
+__be32
+nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
+	__be32 status;
+
+	dprintk("--> %s entry %p\n", __func__, entry);
+
+
+	if (!nfsd41_copy_replay_data(resp, entry)) {
+		/*
+		 * Not enough room to use the replay rpc header, send the
+		 * cached header. Release all the allocated result pages.
+		 */
+		svc_free_res_pages(resp->rqstp);
+		nfsd4_move_pages(resp->rqstp->rq_respages, entry->ce_respages,
+			entry->ce_resused);
+	} else {
+		/* Release all but the first allocated result page */
+
+		resp->rqstp->rq_resused--;
+		svc_free_res_pages(resp->rqstp);
+
+		nfsd4_move_pages(&resp->rqstp->rq_respages[1],
+				 &entry->ce_respages[1],
+				 entry->ce_resused - 1);
+	}
+
+	resp->rqstp->rq_resused = entry->ce_resused;
+	status = entry->ce_status;
+
+	return status;
+}
+
 /*
  * Set the exchange_id flags returned by the server.
  */
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index ef0a368..b5168d1 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -515,6 +515,10 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 *statp)
 		+ rqstp->rq_res.head[0].iov_len;
 	rqstp->rq_res.head[0].iov_len += sizeof(__be32);
 
+	/* NFSv4.1 DRC requires statp */
+	if (rqstp->rq_vers == 4)
+		nfsd4_set_statp(rqstp, statp);
+
 	/* Now call the procedure handler, and encode NFS status. */
 	nfserr = proc->pc_func(rqstp, rqstp->rq_argp, rqstp->rq_resp);
 	nfserr = map_new_errors(rqstp->rq_vers, nfserr);
diff --git a/include/linux/nfsd/cache.h b/include/linux/nfsd/cache.h
index 04b355c..57a83c7 100644
--- a/include/linux/nfsd/cache.h
+++ b/include/linux/nfsd/cache.h
@@ -75,5 +75,10 @@ int	nfsd_reply_cache_init(void);
 void	nfsd_reply_cache_shutdown(void);
 int	nfsd_cache_lookup(struct svc_rqst *, int);
 void	nfsd_cache_update(struct svc_rqst *, int, __be32 *);
+#ifdef CONFIG_NFSD_V4_1
+void	nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp);
+#else /* CONFIG_NFSD_V4_1 */
+static inline void nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp) {}
+#endif /* CONFIG_NFSD_V4_1 */
 
 #endif /* NFSCACHE_H */
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index feab6ec..8ca6a82 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -99,10 +99,23 @@ struct nfs4_callback {
 	struct rpc_clnt *       cb_client;
 };
 
+/* Maximum number of pages per slot cache entry */
+#define NFSD_PAGES_PER_SLOT	1
+
+struct nfsd4_cache_entry {
+	__be32		ce_status;
+	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
+	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
+	short		ce_resused;
+	int		ce_opcnt;
+	int		ce_rpchdrlen;
+};
+
 struct nfsd4_slot {
 	bool				sl_inuse;
 	struct nfsd4_session		*sl_session;
 	u32				sl_seqid;
+	struct nfsd4_cache_entry	sl_cache_entry;
 };
 
 struct nfsd4_session {
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 9e4d8db..cde8947 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -50,6 +50,8 @@ struct nfsd4_compound_state {
 	struct nfs4_stateowner	*replay_owner;
 	/* For sessions DRC */
 	struct nfsd4_slot	*slot;
+	__be32			*statp;
+	u32			status;
 };
 
 struct nfsd4_change_info {
@@ -490,6 +492,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 		struct nfsd4_setclientid_confirm *setclientid_confirm);
 #if defined(CONFIG_NFSD_V4_1)
+extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
+extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
 extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 struct nfsd4_exchange_id *);
-- 
1.6.2.1

[PATCH 21/47] nfsd41: hard page limit for DRC

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Use no more than 1/128th of the number of free pages at nfsd startup for the
v4.1 DRC.

This is an arbitrary default which should probably end up under the control
of an administrator.

Signed-off-by: Andy Adamson <andros@netapp.com>
[moved added fields in struct svc_serv under CONFIG_NFSD_V4_1]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[fix set_max_drc calculation of sv_drc_max_pages]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfssvc.c           |   25 +++++++++++++++++++++++++
 include/linux/nfsd/nfsd.h  |    7 +++++++
 include/linux/sunrpc/svc.h |    4 ++++
 3 files changed, 36 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index b5168d1..ab60320 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -22,6 +22,7 @@
 #include <linux/freezer.h>
 #include <linux/fs_struct.h>
 #include <linux/kthread.h>
+#include <linux/swap.h>
 
 #include <linux/sunrpc/types.h>
 #include <linux/sunrpc/stats.h>
@@ -197,6 +198,28 @@ void nfsd_reset_versions(void)
 	}
 }
 
+/*
+ * Each session guarantees a negotiated per slot memory cache for replies
+ * which in turn consumes memory beyond the v2/v3/v4.0 server. A dedicated
+ * NFSv4.1 server might want to use more memory for a DRC than a machine
+ * with mutiple services.
+ *
+ * Impose a hard limit on the number of pages for the DRC which varies
+ * according to the machines free pages. This is of course only a default.
+ *
+ * For now this is a #defined shift which could be under admin control
+ * in the future.
+ */
+static void set_max_drc(void)
+{
+#ifdef CONFIG_NFSD_V4_1
+	nfsd_serv->sv_drc_max_pages = nr_free_buffer_pages()
+						>> NFSD_DRC_SIZE_SHIFT;
+	nfsd_serv->sv_drc_pages_used = 0;
+	dprintk("%s svc_drc_max_pages %u\n", __func__,
+		nfsd_serv->sv_drc_max_pages);
+#endif /* CONFIG_NFSD_V4_1 */
+}
 
 int nfsd_create_serv(void)
 {
@@ -229,6 +252,8 @@ int nfsd_create_serv(void)
 				      nfsd_last_thread, nfsd, THIS_MODULE);
 	if (nfsd_serv == NULL)
 		err = -ENOMEM;
+	else
+		set_max_drc();
 
 	do_gettimeofday(&nfssvc_boot);		/* record boot time */
 	return err;
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index ab9616d..3bccec3 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -361,6 +361,13 @@ extern struct timeval	nfssvc_boot;
  | FATTR4_WORD1_TIME_DELTA   | FATTR4_WORD1_TIME_METADATA    \
  | FATTR4_WORD1_TIME_MODIFY     | FATTR4_WORD1_TIME_MODIFY_SET | FATTR4_WORD1_MOUNTED_ON_FILEID)
 
+#if defined(CONFIG_NFSD_V4_1)
+
+/* The percent of nr_free_buffer_pages used by the V4.1 server DRC */
+#define NFSD_DRC_SIZE_SHIFT	7
+
+#endif /* CONFIG_NFSD_V4_1 */
+
 /* These will return ERR_INVAL if specified in GETATTR or READDIR. */
 #define NFSD_WRITEONLY_ATTRS_WORD1							    \
 (FATTR4_WORD1_TIME_ACCESS_SET   | FATTR4_WORD1_TIME_MODIFY_SET)
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 815dd58..c4f0e1e 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -95,6 +95,10 @@ struct svc_serv {
 	struct module *		sv_module;	/* optional module to count when
 						 * adding threads */
 	svc_thread_fn		sv_function;	/* main function for threads */
+#if defined(CONFIG_NFSD_V4_1)
+	unsigned int		sv_drc_max_pages; /* Total pages for DRC */
+	unsigned int		sv_drc_pages_used;/* DRC pages used */
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 /*
-- 
1.6.2.1

[PATCH 22/47] nfsd41: nfsd DRC logic

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Replay a request in nfsd4_sequence.
Add a minorversion to struct nfsd4_compound_state.

Pass the current slot to nfs4svc_encode_compound res via struct
nfsd4_compoundres to set an NFSv4.1 DRC entry.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |    7 +++++++
 fs/nfsd/nfs4state.c       |    6 ++++++
 fs/nfsd/nfs4xdr.c         |   13 +++++++++++++
 include/linux/nfsd/xdr4.h |    1 +
 4 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index e703ac2..bdbeb87 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -920,6 +920,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 			BUG_ON(op->status == nfs_ok);
 
 encode_op:
+		/* Only from SEQUENCE or CREATE_SESSION */
+		if (resp->cstate.status == nfserr_replay_cache) {
+			dprintk("%s NFS4.1 replay from cache\n", __func__);
+			status = op->status;
+			goto out;
+		}
 		if (op->status == nfserr_replay_me) {
 			op->replay = &cstate->replay_owner->so_replay;
 			nfsd4_encode_replay(resp, op);
@@ -948,6 +954,7 @@ encode_op:
 		status = nfserr_jukebox;
 	}
 
+	resp->cstate.status = status;
 	fh_put(&resp->cstate.current_fh);
 	fh_put(&resp->cstate.save_fh);
 	BUG_ON(resp->cstate.replay_owner);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f0ce639..07c869d 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -997,6 +997,8 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
 	}
 
 	resp->rqstp->rq_resused = entry->ce_resused;
+	resp->opcnt = entry->ce_opcnt;
+	resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
 	status = entry->ce_status;
 
 	return status;
@@ -1217,6 +1219,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	status = check_slot_seqid(seq->seqid, slot);
 	if (status == nfserr_replay_cache) {
 		cstate->slot = slot;
+		/* Return the cached reply status and set cstate->status
+		 * for nfsd4_svc_encode_compoundres processing*/
+		status = nfsd4_replay_cache_entry(resp);
+		cstate->status = nfserr_replay_cache;
 		goto replay_cache;
 	}
 	if (status)
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index c6b490e..57afb33 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3059,6 +3059,19 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 		iov = &rqstp->rq_res.head[0];
 	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
 	BUG_ON(iov->iov_len > PAGE_SIZE);
+#ifdef CONFIG_NFSD_V4_1
+	if (resp->cstate.slot != NULL) {
+		if (resp->cstate.status == nfserr_replay_cache) {
+			iov->iov_len = resp->cstate.iovlen;
+		} else {
+			nfsd4_set_cache_entry(resp);
+			dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
+			resp->cstate.slot->sl_inuse = 0;
+		}
+		if (resp->cstate.slot->sl_session)
+			nfsd4_put_session(resp->cstate.slot->sl_session);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 	return 1;
 }
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index cde8947..5c0d376 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -51,6 +51,7 @@ struct nfsd4_compound_state {
 	/* For sessions DRC */
 	struct nfsd4_slot	*slot;
 	__be32			*statp;
+	size_t			iovlen;
 	u32			status;
 };
 
-- 
1.6.2.1

[PATCH 23/47] nfsd41: clear DRC cache on free_session

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 07c869d..37865c9 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -442,12 +442,19 @@ release_session(struct nfsd4_session *ses)
 	nfsd4_put_session(ses);
 }
 
+static void nfsd4_release_respages(struct page **respages, short resused);
+
 void
 free_session(struct kref *kref)
 {
 	struct nfsd4_session *ses;
+	int i;
 
 	ses = container_of(kref, struct nfsd4_session, se_ref);
+	for (i = 0; i < ses->se_fnumslots; i++) {
+		struct nfsd4_cache_entry *e = &ses->se_slots[i].sl_cache_entry;
+		nfsd4_release_respages(e->ce_respages, e->ce_resused);
+	}
 	kfree(ses->se_slots);
 	kfree(ses);
 }
-- 
1.6.2.1

[PATCH 24/47] nfsd41: create_session operation

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the create_session operation confoming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

Look up the client id (generated by the server on exchange_id,
given by the client on create_session).
If neither a confirmed or unconfirmed client is found
then the client id is stale
If a confirmed cilent is found (i.e. we already received
create_session for it) then compare the sequence id
to determine if it's a replay or possibly a mis-ordered rpc.
If the seqid is in order, update the confirmed client seqid
and procedd with updating the session parameters.

If an unconfirmed client_id is found then verify the creds
and seqid.  If both match move the client id to confirmed state
and proceed with processing the create_session.

Currently, we do not support persistent sessions, and RDMA.

alloc_init_session generates a new sessionid and creates
a session structure.

NFSD_PAGES_PER_SLOT is used for the max response cached calculation, and for
the counting of DRC pages using the hard limits set in struct srv_serv.

A note on NFSD_PAGES_PER_SLOT:

Other patches in this series allow for NFSD_PAGES_PER_SLOT + 1 pages to be
cached in a DRC slot when the response size is less than NFSD_PAGES_PER_SLOT *
PAGE_SIZE but xdr_buf pages are used. e.g. a READDIR operation will encode a
small amount of data in the xdr_buf head, and then the READDIR in the xdr_buf
pages.  So, the hard limit calculation use of pages by a session is
underestimated by the number of cached operations using the xdr_buf pages.

Yet another patch caches no pages for the solo sequence operation, or any
compound where cache_this is False.  So the hard limit calculation use of
pages by a session is overestimated by the number of these operations in the
cache.

TODO: improve resource pre-allocation and negotiate session
parameters accordingly.  Respect and possibly adjust
backchannel attributes.

Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
Signed-off-by: Dean Hildebrand <dhildeb@us.ibm.com>
[nfsd41: remove headerpadsz from channel attributes]
Our client and server only support a headerpadsz of 0.
[nfsd41: use DRC limits in fore channel init]
[nfsd41: do not change CREATE_SESSION back channel attrs]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[use sessionid_lock spin lock]
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |  197 +++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c          |  147 ++++++++++++++++++++++++++++++++-
 include/linux/nfsd/state.h |    7 ++
 include/linux/nfsd/xdr4.h  |   21 +++++-
 4 files changed, 368 insertions(+), 4 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 37865c9..e4e2c19 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -68,6 +68,9 @@ static u32 current_delegid = 1;
 static u32 nfs4_init;
 static stateid_t zerostateid;             /* bits all 0 */
 static stateid_t onestateid;              /* bits all 1 */
+#ifdef CONFIG_NFSD_V4_1
+static u64 current_sessionid = 1;
+#endif /* CONFIG_NFSD_V4_1 */
 
 #define ZERO_STATEID(stateid) (!memcmp((stateid), &zerostateid, sizeof(stateid_t)))
 #define ONE_STATEID(stateid)  (!memcmp((stateid), &onestateid, sizeof(stateid_t)))
@@ -402,6 +405,138 @@ dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
 	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
 }
 
+static void
+gen_sessionid(struct nfsd4_session *ses)
+{
+	struct nfs4_client *clp = ses->se_client;
+	struct nfsd4_sessionid *sid;
+
+	sid = (struct nfsd4_sessionid *)ses->se_sessionid.data;
+	sid->clientid = clp->cl_clientid;
+	sid->sequence = current_sessionid++;
+	sid->reserved = 0;
+}
+
+/*
+ * Give the client the number of slots it requests bound by
+ * NFSD_MAX_SLOTS_PER_SESSION and by sv_drc_max_pages.
+ *
+ * If we run out of pages (sv_drc_pages_used == sv_drc_max_pages) we
+ * should (up to a point) re-negotiate active sessions and reduce their
+ * slot usage to make rooom for new connections. For now we just fail the
+ * create session.
+ */
+static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
+{
+	int status = 0, np = fchan->maxreqs * NFSD_PAGES_PER_SLOT;
+
+	spin_lock(&nfsd_serv->sv_lock);
+	if (np + nfsd_serv->sv_drc_pages_used > nfsd_serv->sv_drc_max_pages)
+		np = nfsd_serv->sv_drc_max_pages - nfsd_serv->sv_drc_pages_used;
+	nfsd_serv->sv_drc_pages_used += np;
+	spin_unlock(&nfsd_serv->sv_lock);
+
+	if (np <= 0) {
+		status = nfserr_resource;
+		fchan->maxreqs = 0;
+	} else
+		fchan->maxreqs = np / NFSD_PAGES_PER_SLOT;
+
+	return status;
+}
+
+/*
+ * fchan holds the client values on input, and the server values on output
+ */
+static int init_forechannel_attrs(struct svc_rqst *rqstp,
+				    struct nfsd4_session *session,
+				    struct nfsd4_channel_attrs *fchan)
+{
+	int status = 0;
+	__u32   maxcount = svc_max_payload(rqstp);
+
+	/* headerpadsz set to zero in encode routine*/
+
+	/* Use the client's max request and max response size if possible */
+	if (fchan->maxreq_sz > maxcount)
+		fchan->maxreq_sz = maxcount;
+	session->se_fmaxreq_sz = fchan->maxreq_sz;
+
+	if (fchan->maxresp_sz > maxcount)
+		fchan->maxresp_sz = maxcount;
+	session->se_fmaxresp_sz = fchan->maxresp_sz;
+
+	/* Set the max response cached size our default which is
+	 * a multiple of PAGE_SIZE and small */
+	session->se_fmaxresp_cached = NFSD_PAGES_PER_SLOT * PAGE_SIZE;
+	fchan->maxresp_cached = session->se_fmaxresp_cached;
+
+	/* Use the client's maxops if possible */
+	if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND)
+		fchan->maxops = NFSD_MAX_OPS_PER_COMPOUND;
+	session->se_fmaxops = fchan->maxops;
+
+	/* try to use the client requested number of slots */
+	if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
+		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
+
+	/* FIXME: Error means no more DRC pages so the server should
+	 * recover pages from existing sessions. For now fail session
+	 * creation.
+	 */
+	status = set_forechannel_maxreqs(fchan);
+
+	session->se_fnumslots = fchan->maxreqs;
+	return status;
+}
+
+static int
+alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
+		   struct nfsd4_create_session *cses)
+{
+	struct nfsd4_session *new;
+	int idx, status = nfserr_resource, slotsize, i;
+
+	new = kzalloc(sizeof(*new), GFP_KERNEL);
+	if (!new)
+		goto out;
+
+	/* FIXME: For now, we just accept the client back channel attributes. */
+	status = init_forechannel_attrs(rqstp, new, &cses->fore_channel);
+	if (status)
+		goto out_free;
+
+	slotsize = new->se_fnumslots * sizeof(struct nfsd4_slot);
+	new->se_slots = kzalloc(slotsize, GFP_KERNEL);
+	if (!new->se_slots)
+		goto out_free;
+
+	for (i = 0; i < new->se_fnumslots; i++)
+		new->se_slots[i].sl_session = new;
+
+	new->se_client = clp;
+	gen_sessionid(new);
+	idx = hash_sessionid(&new->se_sessionid);
+	memcpy(clp->cl_sessionid.data, new->se_sessionid.data,
+	       NFS4_MAX_SESSIONID_LEN);
+
+	new->se_flags = cses->flags;
+	kref_init(&new->se_ref);
+	INIT_LIST_HEAD(&new->se_hash);
+	INIT_LIST_HEAD(&new->se_perclnt);
+	spin_lock(&sessionid_lock);
+	list_add(&new->se_hash, &sessionid_hashtbl[idx]);
+	list_add(&new->se_perclnt, &clp->cl_sessions);
+	spin_unlock(&sessionid_lock);
+
+	status = nfs_ok;
+out:
+	return status;
+out_free:
+	kfree(new);
+	goto out;
+}
+
 /* caller must hold sessionid_lock */
 static struct nfsd4_session *
 find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
@@ -1186,7 +1321,67 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		     struct nfsd4_compound_state *cstate,
 		     struct nfsd4_create_session *cr_ses)
 {
-	return -1;	/* stub */
+	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
+	struct nfs4_client *conf, *unconf;
+	int status = 0;
+
+	nfs4_lock_state();
+	unconf = find_unconfirmed_client(&cr_ses->clientid);
+	conf = find_confirmed_client(&cr_ses->clientid);
+
+	if (conf) {
+		status = nfs_ok;
+		if (conf->cl_seqid == cr_ses->seqid) {
+			dprintk("Got a create_session replay! seqid= %d\n",
+				conf->cl_seqid);
+			goto out_replay;
+		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
+			status = nfserr_seq_misordered;
+			dprintk("Sequence misordered!\n");
+			dprintk("Expected seqid= %d but got seqid= %d\n",
+				conf->cl_seqid, cr_ses->seqid);
+			goto out;
+		}
+		conf->cl_seqid++;
+	} else if (unconf) {
+		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
+		    (ip_addr != unconf->cl_addr)) {
+			status = nfserr_clid_inuse;
+			goto out;
+		}
+
+		if (unconf->cl_seqid != cr_ses->seqid) {
+			status = nfserr_seq_misordered;
+			goto out;
+		}
+
+		move_to_confirmed(unconf);
+
+		/*
+		 * We do not support RDMA or persistent sessions
+		 */
+		cr_ses->flags &= ~SESSION4_PERSIST;
+		cr_ses->flags &= ~SESSION4_RDMA;
+
+		conf = unconf;
+	} else {
+		status = nfserr_stale_clientid;
+		goto out;
+	}
+
+	status = alloc_init_session(rqstp, conf, cr_ses);
+	if (status)
+		goto out;
+
+out_replay:
+	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
+	       NFS4_MAX_SESSIONID_LEN);
+	cr_ses->seqid = conf->cl_seqid;
+
+out:
+	nfs4_unlock_state();
+	dprintk("%s returns %d\n", __func__, ntohl(status));
+	return status;
 }
 
 __be32
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 57afb33..60db854 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1100,7 +1100,108 @@ static __be32
 nfsd4_decode_create_session(struct nfsd4_compoundargs *argp,
 			    struct nfsd4_create_session *sess)
 {
-	return nfserr_opnotsupp;	/* stub */
+	DECODE_HEAD;
+
+	u32 dummy;
+	char *machine_name;
+	int i;
+	int nr_secflavs;
+
+	READ_BUF(16);
+	COPYMEM(&sess->clientid, 8);
+	READ32(sess->seqid);
+	READ32(sess->flags);
+
+	/* Fore channel attrs */
+	READ_BUF(28);
+	READ32(dummy); /* headerpadsz is always 0 */
+	READ32(sess->fore_channel.maxreq_sz);
+	READ32(sess->fore_channel.maxresp_sz);
+	READ32(sess->fore_channel.maxresp_cached);
+	READ32(sess->fore_channel.maxops);
+	READ32(sess->fore_channel.maxreqs);
+	READ32(sess->fore_channel.nr_rdma_attrs);
+	if (sess->fore_channel.nr_rdma_attrs == 1) {
+		READ_BUF(4);
+		READ32(sess->fore_channel.rdma_attrs);
+	} else if (sess->fore_channel.nr_rdma_attrs > 1) {
+		dprintk("Too many fore channel attr bitmaps!\n");
+		goto xdr_error;
+	}
+
+	/* Back channel attrs */
+	READ_BUF(28);
+	READ32(dummy); /* headerpadsz is always 0 */
+	READ32(sess->back_channel.maxreq_sz);
+	READ32(sess->back_channel.maxresp_sz);
+	READ32(sess->back_channel.maxresp_cached);
+	READ32(sess->back_channel.maxops);
+	READ32(sess->back_channel.maxreqs);
+	READ32(sess->back_channel.nr_rdma_attrs);
+	if (sess->back_channel.nr_rdma_attrs == 1) {
+		READ_BUF(4);
+		READ32(sess->back_channel.rdma_attrs);
+	} else if (sess->back_channel.nr_rdma_attrs > 1) {
+		dprintk("Too many back channel attr bitmaps!\n");
+		goto xdr_error;
+	}
+
+	READ_BUF(8);
+	READ32(sess->callback_prog);
+
+	/* callback_sec_params4 */
+	READ32(nr_secflavs);
+	for (i = 0; i < nr_secflavs; ++i) {
+		READ_BUF(4);
+		READ32(dummy);
+		switch (dummy) {
+		case RPC_AUTH_NULL:
+			/* Nothing to read */
+			break;
+		case RPC_AUTH_UNIX:
+			READ_BUF(8);
+			/* stamp */
+			READ32(dummy);
+
+			/* machine name */
+			READ32(dummy);
+			READ_BUF(dummy);
+			SAVEMEM(machine_name, dummy);
+
+			/* uid, gid */
+			READ_BUF(8);
+			READ32(sess->uid);
+			READ32(sess->gid);
+
+			/* more gids */
+			READ_BUF(4);
+			READ32(dummy);
+			READ_BUF(dummy * 4);
+			for (i = 0; i < dummy; ++i)
+				READ32(dummy);
+			break;
+		case RPC_AUTH_GSS:
+			dprintk("RPC_AUTH_GSS callback secflavor "
+				"not supported!\n");
+			READ_BUF(8);
+			/* gcbp_service */
+			READ32(dummy);
+			/* gcbp_handle_from_server */
+			READ32(dummy);
+			READ_BUF(dummy);
+			p += XDR_QUADLEN(dummy);
+			/* gcbp_handle_from_client */
+			READ_BUF(4);
+			READ32(dummy);
+			READ_BUF(dummy);
+			p += XDR_QUADLEN(dummy);
+			break;
+		default:
+			dprintk("Illegal callback secflavor\n");
+			return nfserr_inval;
+		}
+	}
+	DECODE_TAIL;
 }
 
 static __be32
@@ -2829,7 +2930,49 @@ static __be32
 nfsd4_encode_create_session(struct nfsd4_compoundres *resp, int nfserr,
 			    struct nfsd4_create_session *sess)
 {
-	/* stub */
+	ENCODE_HEAD;
+
+	if (nfserr)
+		goto out;
+
+	RESERVE_SPACE(24);
+	WRITEMEM(sess->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(sess->seqid);
+	WRITE32(sess->flags);
+	ADJUST_ARGS();
+
+	RESERVE_SPACE(28);
+	WRITE32(0); /* headerpadsz */
+	WRITE32(sess->fore_channel.maxreq_sz);
+	WRITE32(sess->fore_channel.maxresp_sz);
+	WRITE32(sess->fore_channel.maxresp_cached);
+	WRITE32(sess->fore_channel.maxops);
+	WRITE32(sess->fore_channel.maxreqs);
+	WRITE32(sess->fore_channel.nr_rdma_attrs);
+	ADJUST_ARGS();
+
+	if (sess->fore_channel.nr_rdma_attrs) {
+		RESERVE_SPACE(4);
+		WRITE32(sess->fore_channel.rdma_attrs);
+		ADJUST_ARGS();
+	}
+
+	RESERVE_SPACE(28);
+	WRITE32(0); /* headerpadsz */
+	WRITE32(sess->back_channel.maxreq_sz);
+	WRITE32(sess->back_channel.maxresp_sz);
+	WRITE32(sess->back_channel.maxresp_cached);
+	WRITE32(sess->back_channel.maxops);
+	WRITE32(sess->back_channel.maxreqs);
+	WRITE32(sess->back_channel.nr_rdma_attrs);
+	ADJUST_ARGS();
+
+	if (sess->back_channel.nr_rdma_attrs) {
+		RESERVE_SPACE(4);
+		WRITE32(sess->back_channel.rdma_attrs);
+		ADJUST_ARGS();
+	}
+out:
 	return nfserr;
 }
 
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 8ca6a82..98d7b1c 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -99,8 +99,12 @@ struct nfs4_callback {
 	struct rpc_clnt *       cb_client;
 };
 
+/* Maximum number of slots per session. 128 is useful for long haul TCP */
+#define NFSD_MAX_SLOTS_PER_SESSION	128
 /* Maximum number of pages per slot cache entry */
 #define NFSD_PAGES_PER_SLOT	1
+/* Maximum number of operations per session compound */
+#define NFSD_MAX_OPS_PER_COMPOUND	16
 
 struct nfsd4_cache_entry {
 	__be32		ce_status;
@@ -188,6 +192,9 @@ struct nfs4_client {
 	struct list_head	cl_sessions;
 	u32			cl_seqid;       /* seqid for create_session */
 	u32			cl_exchange_flags;
+	struct nfs4_sessionid	cl_sessionid;
+
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 5c0d376..c7bf0a1 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -360,8 +360,27 @@ struct nfsd4_exchange_id {
 	int		spa_how;
 };
 
+struct nfsd4_channel_attrs {
+	u32		headerpadsz;
+	u32		maxreq_sz;
+	u32		maxresp_sz;
+	u32		maxresp_cached;
+	u32		maxops;
+	u32		maxreqs;
+	u32		nr_rdma_attrs;
+	u32		rdma_attrs;
+};
+
 struct nfsd4_create_session {
-	int	foo;	/* stub */
+	clientid_t		clientid;
+	struct nfs4_sessionid	sessionid;
+	u32			seqid;
+	u32			flags;
+	struct nfsd4_channel_attrs fore_channel;
+	struct nfsd4_channel_attrs back_channel;
+	u32			callback_prog;
+	u32			uid;
+	u32			gid;
 };
 
 struct nfsd4_sequence {
-- 
1.6.2.1

[PATCH 25/47] nfsd41: Add a create session replay cache

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Replace the nfs4_client cl_seqid field with a single struct nfs41_slot used
for the create session replay cache.

The CREATE_SESSION slot sets the sl_session pointer to NULL. Otherwise, the
slot and it's replay cache are used just like the session slots.

Fix unconfirmed create_session replay response by initializing the
create_session slot sequence id to 0.

A future patch will set the CREATE_SESSION cache when a SEQUENCE operation
preceeds the CREATE_SESSION operation. This compound is currently only cached
in the session slot table.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++-------------
 include/linux/nfsd/state.h |    2 +-
 2 files changed, 34 insertions(+), 15 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index e4e2c19..61af434 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -662,6 +662,10 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+#if defined(CONFIG_NFSD_V4_1)
+	nfsd4_release_respages(clp->cl_slot.sl_cache_entry.ce_respages,
+			     clp->cl_slot.sl_cache_entry.ce_resused);
+#endif /* CONFIG_NFSD_V4_1 */
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1055,12 +1059,14 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
 {
 	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
 	struct svc_rqst *rqstp = resp->rqstp;
+	struct nfsd4_compoundargs *args = rqstp->rq_argp;
+	struct nfsd4_op *op = &args->ops[resp->opcnt];
 	struct kvec *resv = &rqstp->rq_res.head[0];
 
 	dprintk("--> %s entry %p\n", __func__, entry);
 
-	/* Don't cache a failed OP_SEQUENCE */
-	if (resp->opcnt == 1 && resp->cstate.status)
+	/* Don't cache a failed OP_SEQUENCE. */
+	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
 		return;
 	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
 	entry->ce_resused = rqstp->rq_resused;
@@ -1276,11 +1282,12 @@ out_copy:
 	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
 	exid->clientid.cl_id = new->cl_clientid.cl_id;
 
-	new->cl_seqid = exid->seqid = 1;
+	new->cl_slot.sl_seqid = 0;
+	exid->seqid = 1;
 	nfsd4_set_ex_flags(new, exid);
 
 	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
-		new->cl_seqid, new->cl_exchange_flags);
+		new->cl_slot.sl_seqid, new->cl_exchange_flags);
 	status = nfs_ok;
 
 out:
@@ -1322,7 +1329,9 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		     struct nfsd4_create_session *cr_ses)
 {
 	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfs4_client *conf, *unconf;
+	struct nfsd4_slot *slot = NULL;
 	int status = 0;
 
 	nfs4_lock_state();
@@ -1330,19 +1339,24 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 	conf = find_confirmed_client(&cr_ses->clientid);
 
 	if (conf) {
-		status = nfs_ok;
-		if (conf->cl_seqid == cr_ses->seqid) {
+		slot = &conf->cl_slot;
+		status = check_slot_seqid(cr_ses->seqid, slot);
+		if (status == nfserr_replay_cache) {
 			dprintk("Got a create_session replay! seqid= %d\n",
-				conf->cl_seqid);
-			goto out_replay;
-		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
+				slot->sl_seqid);
+			cstate->slot = slot;
+			cstate->status = status;
+			/* Return the cached reply status */
+			status = nfsd4_replay_cache_entry(resp);
+			goto out;
+		} else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
 			status = nfserr_seq_misordered;
 			dprintk("Sequence misordered!\n");
 			dprintk("Expected seqid= %d but got seqid= %d\n",
-				conf->cl_seqid, cr_ses->seqid);
+				slot->sl_seqid, cr_ses->seqid);
 			goto out;
 		}
-		conf->cl_seqid++;
+		conf->cl_slot.sl_seqid++;
 	} else if (unconf) {
 		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
 		    (ip_addr != unconf->cl_addr)) {
@@ -1350,11 +1364,15 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 			goto out;
 		}
 
-		if (unconf->cl_seqid != cr_ses->seqid) {
+		slot = &unconf->cl_slot;
+		status = check_slot_seqid(cr_ses->seqid, slot);
+		if (status) {
+			/* an unconfirmed replay returns misordered */
 			status = nfserr_seq_misordered;
 			goto out;
 		}
 
+		slot->sl_seqid++; /* from 0 to 1 */
 		move_to_confirmed(unconf);
 
 		/*
@@ -1373,11 +1391,12 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 	if (status)
 		goto out;
 
-out_replay:
 	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
 	       NFS4_MAX_SESSIONID_LEN);
-	cr_ses->seqid = conf->cl_seqid;
+	cr_ses->seqid = slot->sl_seqid;
 
+	slot->sl_inuse = true;
+	cstate->slot = slot;
 out:
 	nfs4_unlock_state();
 	dprintk("%s returns %d\n", __func__, ntohl(status));
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 98d7b1c..49d89fd 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -190,7 +190,7 @@ struct nfs4_client {
 	u32			cl_firststate;	/* recovery dir creation */
 #ifdef CONFIG_NFSD_V4_1
 	struct list_head	cl_sessions;
-	u32			cl_seqid;       /* seqid for create_session */
+	struct nfsd4_slot	cl_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
 
-- 
1.6.2.1

[PATCH 26/47] nfsd41: non-page DRC for solo sequence responses

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

A session inactivity time compound (lease renewal) or a compound where the
sequence operation has sa_cachethis set to FALSE do not require any pages
to be held in the v4.1 DRC. This is because struct nfsd4_slot is already
caching the session information.

Add logic to the nfs41 server to not cache response pages for solo sequence
responses.

Return nfserr_replay_uncached_rep on the operation following the sequence
operation when sa_cachethis is FALSE.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c         |   34 +++++++++++++++++++++++++++++-
 fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++++++++++-----
 fs/nfsd/nfs4xdr.c          |    5 ++-
 include/linux/nfsd/state.h |    1 +
 include/linux/nfsd/xdr4.h  |   15 +++++++++++++-
 5 files changed, 91 insertions(+), 11 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index bdbeb87..a273023 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -828,6 +828,34 @@ static struct nfsd4_operation nfsd4_ops[];
 static const char *nfsd4_op_name(unsigned opnum);
 
 /*
+ * This is a replay of a compound for which no cache entry pages
+ * were used. Encode the sequence operation, and if cachethis is FALSE
+ * encode the uncache rep error on the next operation.
+ */
+static __be32
+nfsd4_enc_no_page_replay(struct nfsd4_compoundargs *args,
+			 struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_op *op;
+
+	dprintk("--> %s resp->opcnt %d ce_cachethis %u \n", __func__,
+		resp->opcnt, resp->cstate.slot->sl_cache_entry.ce_cachethis);
+
+	/* Encode the replayed sequence operation */
+	BUG_ON(resp->opcnt != 1);
+	op = &args->ops[resp->opcnt - 1];
+	nfsd4_encode_operation(resp, op);
+
+	/*return nfserr_retry_uncached_rep in next operation. */
+	if (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0) {
+		op = &args->ops[resp->opcnt++];
+		op->status = nfserr_retry_uncached_rep;
+		nfsd4_encode_operation(resp, op);
+	}
+	return op->status;
+}
+
+/*
  * COMPOUND call.
  */
 static __be32
@@ -879,7 +907,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 		dprintk("nfsv4 compound op #%d/%d: %d (%s)\n",
 			resp->opcnt, args->opcnt, op->opnum,
 			nfsd4_op_name(op->opnum));
-
 		/*
 		 * The XDR decode routines may have pre-set op->status;
 		 * for example, if there is a miscellaneous XDR error
@@ -923,7 +950,10 @@ encode_op:
 		/* Only from SEQUENCE or CREATE_SESSION */
 		if (resp->cstate.status == nfserr_replay_cache) {
 			dprintk("%s NFS4.1 replay from cache\n", __func__);
-			status = op->status;
+			if (nfsd4_no_page_in_cache(resp))
+				status = nfsd4_enc_no_page_replay(args, resp);
+			else
+				status = op->status;
 			goto out;
 		}
 		if (op->status == nfserr_replay_me) {
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 61af434..f42cda9 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1068,17 +1068,31 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
 	/* Don't cache a failed OP_SEQUENCE. */
 	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
 		return;
+
 	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
+	entry->ce_opcnt = resp->opcnt;
+	entry->ce_status = resp->cstate.status;
+
+	/*
+	 * Don't need a page to cache just the sequence operation - the slot
+	 * does this for us!
+	 */
+
+	if (nfsd4_no_page_in_cache(resp)) {
+		entry->ce_resused = 0;
+		entry->ce_rpchdrlen = 0;
+		dprintk("%s Just cache SEQUENCE. ce_cachethis %d\n", __func__,
+			resp->cstate.slot->sl_cache_entry.ce_cachethis);
+		return;
+	}
 	entry->ce_resused = rqstp->rq_resused;
 	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
 		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
 	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
 			 entry->ce_resused);
-	entry->ce_status = resp->cstate.status;
 	entry->ce_datav.iov_base = resp->cstate.statp;
 	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
 				(char *)page_address(rqstp->rq_respages[0]));
-	entry->ce_opcnt = resp->opcnt;
 	/* Current request rpc header length*/
 	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
 				(char *)page_address(rqstp->rq_respages[0]);
@@ -1117,13 +1131,28 @@ nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
  * cached page.  Replace any futher replay pages from the cache.
  */
 __be32
-nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
+nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
+			 struct nfsd4_sequence *seq)
 {
 	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
 	__be32 status;
 
 	dprintk("--> %s entry %p\n", __func__, entry);
 
+	/*
+	 * If this is just the sequence operation, we did not keep
+	 * a page in the cache entry because we can just use the
+	 * slot info stored in struct nfsd4_sequence that was checked
+	 * against the slot in nfsd4_sequence().
+	 *
+	 * This occurs when seq->cachethis is FALSE, or when the client
+	 * session inactivity timer fires and a solo sequence operation
+	 * is sent (lease renewal).
+	 */
+	if (seq && nfsd4_no_page_in_cache(resp)) {
+		seq->maxslots = resp->cstate.slot->sl_session->se_fnumslots;
+		return nfs_ok;
+	}
 
 	if (!nfsd41_copy_replay_data(resp, entry)) {
 		/*
@@ -1347,7 +1376,7 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 			cstate->slot = slot;
 			cstate->status = status;
 			/* Return the cached reply status */
-			status = nfsd4_replay_cache_entry(resp);
+			status = nfsd4_replay_cache_entry(resp, NULL);
 			goto out;
 		} else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
 			status = nfserr_seq_misordered;
@@ -1397,6 +1426,8 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 
 	slot->sl_inuse = true;
 	cstate->slot = slot;
+	/* Ensure a page is used for the cache */
+	slot->sl_cache_entry.ce_cachethis = 1;
 out:
 	nfs4_unlock_state();
 	dprintk("%s returns %d\n", __func__, ntohl(status));
@@ -1441,8 +1472,8 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	if (status == nfserr_replay_cache) {
 		cstate->slot = slot;
 		/* Return the cached reply status and set cstate->status
-		 * for nfsd4_svc_encode_compoundres processing*/
-		status = nfsd4_replay_cache_entry(resp);
+		 * for nfsd4_svc_encode_compoundres processing */
+		status = nfsd4_replay_cache_entry(resp, seq);
 		cstate->status = nfserr_replay_cache;
 		goto replay_cache;
 	}
@@ -1452,6 +1483,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	/* Success! bump slot seqid */
 	slot->sl_inuse = true;
 	slot->sl_seqid = seq->seqid;
+	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
+	/* Always set the cache entry cachethis for solo sequence */
+	if (nfsd4_is_solo_sequence(resp))
+		slot->sl_cache_entry.ce_cachethis = 1;
 
 	cstate->slot = slot;
 
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 60db854..a8bb04a 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -2984,7 +2984,7 @@ nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
 	return nfserr;
 }
 
-static __be32
+__be32
 nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
 		      struct nfsd4_sequence *seq)
 {
@@ -3204,7 +3204,8 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 	BUG_ON(iov->iov_len > PAGE_SIZE);
 #ifdef CONFIG_NFSD_V4_1
 	if (resp->cstate.slot != NULL) {
-		if (resp->cstate.status == nfserr_replay_cache) {
+		if (resp->cstate.status == nfserr_replay_cache &&
+				!nfsd4_no_page_in_cache(resp)) {
 			iov->iov_len = resp->cstate.iovlen;
 		} else {
 			nfsd4_set_cache_entry(resp);
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 49d89fd..47c7836 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -110,6 +110,7 @@ struct nfsd4_cache_entry {
 	__be32		ce_status;
 	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
 	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
+	int		ce_cachethis;
 	short		ce_resused;
 	int		ce_opcnt;
 	int		ce_rpchdrlen;
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index c7bf0a1..641e5cd 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -482,6 +482,18 @@ struct nfsd4_compoundres {
 	struct nfsd4_compound_state	cstate;
 };
 
+static inline u32 nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
+	return args->opcnt == 1 ? 1 : 0;
+}
+
+static inline u32 nfsd4_no_page_in_cache(struct nfsd4_compoundres *resp)
+{
+	return (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0 ||
+			nfsd4_is_solo_sequence(resp));
+}
+
 #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
 
 static inline void
@@ -513,7 +525,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 		struct nfsd4_setclientid_confirm *setclientid_confirm);
 #if defined(CONFIG_NFSD_V4_1)
 extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
-extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
+extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
+		struct nfsd4_sequence *seq);
 extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 struct nfsd4_exchange_id *);
-- 
1.6.2.1

[PATCH 27/47] nfsd41: destroy_session operation

[Benny Halevy]

Implement the destory_session operation confoming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

[use sessionid_lock spin lock]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c       |   29 ++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c         |    7 +++++--
 include/linux/nfsd/xdr4.h |    2 +-
 3 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f42cda9..9c93f96 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1439,7 +1439,34 @@ nfsd4_destroy_session(struct svc_rqst *r,
 		      struct nfsd4_compound_state *cstate,
 		      struct nfsd4_destroy_session *sessionid)
 {
-	return -1;	/* stub */
+	struct nfsd4_session *ses;
+	u32 status = nfserr_badsession;
+
+	/* Notes:
+	 * - The confirmed nfs4_client->cl_sessionid holds destroyed sessinid
+	 * - Should we return nfserr_back_chan_busy if waiting for
+	 *   callbacks on to-be-destroyed session?
+	 * - Do we need to clear any callback info from previous session?
+	 */
+
+	dump_sessionid(__func__, &sessionid->sessionid);
+	spin_lock(&sessionid_lock);
+	ses = find_in_sessionid_hashtbl(&sessionid->sessionid);
+	if (!ses) {
+		spin_unlock(&sessionid_lock);
+		goto out;
+	}
+
+	unhash_session(ses);
+	spin_unlock(&sessionid_lock);
+
+	/* wait for callbacks */
+	shutdown_callback_client(ses->se_client);
+	nfsd4_put_session(ses);
+	status = nfs_ok;
+out:
+	dprintk("%s returns %d\n", __func__, ntohl(status));
+	return status;
 }
 
 __be32
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a8bb04a..5720aab 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1208,7 +1208,11 @@ static __be32
 nfsd4_decode_destroy_session(struct nfsd4_compoundargs *argp,
 			     struct nfsd4_destroy_session *destroy_session)
 {
-	return nfserr_opnotsupp;	/* stub */
+	DECODE_HEAD;
+	READ_BUF(NFS4_MAX_SESSIONID_LEN);
+	COPYMEM(destroy_session->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+
+	DECODE_TAIL;
 }
 
 static __be32
@@ -2980,7 +2984,6 @@ static __be32
 nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
 			     struct nfsd4_destroy_session *destroy_session)
 {
-	/* stub */
 	return nfserr;
 }
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 641e5cd..37a7c51 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -396,7 +396,7 @@ struct nfsd4_sequence {
 };
 
 struct nfsd4_destroy_session {
-	int	foo;	/* stub */
+	struct nfs4_sessionid	sessionid;
 };
 #endif /* CONFIG_NFSD_V4_1 */
 
-- 
1.6.2.1

[PATCH 28/47] nfsd41: stateid handling

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

When sessions are used, stateful operation sequenceid and stateid handling
are not used. When sessions are used,  on the first open set the seqid to 1,
mark state confirmed and skip seqid processing.

When sessionas are used the stateid generation number is ignored when it is zero
whereas without sessions bad_stateid or stale stateid is returned.

Add flags to propagate session use to all stateful ops and down to
check_stateid_generation.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
[nfsd4_has_session should return a boolean, not u32]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c         |   17 ++++++++--
 fs/nfsd/nfs4state.c        |   70 ++++++++++++++++++++++++++++++++++---------
 fs/nfsd/nfs4xdr.c          |    2 +-
 include/linux/nfsd/state.h |    1 +
 include/linux/nfsd/xdr4.h  |    8 ++++-
 5 files changed, 77 insertions(+), 21 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index a273023..1d4b2b5 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -179,7 +179,7 @@ nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	nfs4_lock_state();
 
 	/* check seqid for replay. set nfs4_owner */
-	status = nfsd4_process_open1(open);
+	status = nfsd4_process_open1(rqstp, open);
 	if (status == nfserr_replay_me) {
 		struct nfs4_replay *rp = &open->op_stateowner->so_replay;
 		fh_put(&cstate->current_fh);
@@ -504,6 +504,7 @@ nfsd4_read(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	   struct nfsd4_read *read)
 {
 	__be32 status;
+	int flags = RD_STATE;
 
 	/* no need to check permission - this will be done in nfsd_read() */
 
@@ -511,11 +512,13 @@ nfsd4_read(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (read->rd_offset >= OFFSET_MAX)
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	/* check stateid */
 	if ((status = nfs4_preprocess_stateid_op(&cstate->current_fh,
 				&read->rd_stateid,
-				RD_STATE, &read->rd_filp))) {
+				flags, &read->rd_filp))) {
 		dprintk("NFSD: nfsd4_read: couldn't process stateid!\n");
 		goto out;
 	}
@@ -643,11 +646,14 @@ nfsd4_setattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	      struct nfsd4_setattr *setattr)
 {
 	__be32 status = nfs_ok;
+	int flags = WR_STATE;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
 		nfs4_lock_state();
 		status = nfs4_preprocess_stateid_op(&cstate->current_fh,
-			&setattr->sa_stateid, WR_STATE, NULL);
+			&setattr->sa_stateid, flags, NULL);
 		nfs4_unlock_state();
 		if (status) {
 			dprintk("NFSD: nfsd4_setattr: couldn't process stateid!\n");
@@ -679,15 +685,18 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	u32 *p;
 	__be32 status = nfs_ok;
 	unsigned long cnt;
+	int flags = WR_STATE;
 
 	/* no need to check permission - this will be done in nfsd_write() */
 
 	if (write->wr_offset >= OFFSET_MAX)
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	status = nfs4_preprocess_stateid_op(&cstate->current_fh, stateid,
-					WR_STATE, &filp);
+					    flags, &filp);
 	if (filp)
 		get_file(filp);
 	nfs4_unlock_state();
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 9c93f96..bf5b214 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -2199,12 +2199,13 @@ static struct lock_manager_operations nfsd_lease_mng_ops = {
 
 
 __be32
-nfsd4_process_open1(struct nfsd4_open *open)
+nfsd4_process_open1(struct svc_rqst *rqstp, struct nfsd4_open *open)
 {
 	clientid_t *clientid = &open->op_clientid;
 	struct nfs4_client *clp = NULL;
 	unsigned int strhashval;
 	struct nfs4_stateowner *sop = NULL;
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 
 	if (!check_name(open->op_owner))
 		return nfserr_inval;
@@ -2222,6 +2223,9 @@ nfsd4_process_open1(struct nfsd4_open *open)
 			return nfserr_expired;
 		goto renew;
 	}
+	/* When sessions are used, skip open sequenceid processing */
+	if (nfsd4_has_session(&resp->cstate))
+		goto renew;
 	if (!sop->so_confirmed) {
 		/* Replace unconfirmed owners without checking for replay. */
 		clp = sop->so_client;
@@ -2499,6 +2503,7 @@ out:
 __be32
 nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_open *open)
 {
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfs4_file *fp = NULL;
 	struct inode *ino = current_fh->fh_dentry->d_inode;
 	struct nfs4_stateid *stp = NULL;
@@ -2557,9 +2562,14 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 			release_open_stateid(stp);
 			goto out;
 		}
+		if (nfsd4_has_session(&resp->cstate))
+			update_stateid(&stp->st_stateid);
 	}
 	memcpy(&open->op_stateid, &stp->st_stateid, sizeof(stateid_t));
 
+	if (nfsd4_has_session(&resp->cstate))
+		open->op_stateowner->so_confirmed = 1;
+
 	/*
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
@@ -2580,7 +2590,8 @@ out:
 	* To finish the open response, we just need to set the rflags.
 	*/
 	open->op_rflags = NFS4_OPEN_RESULT_LOCKTYPE_POSIX;
-	if (!open->op_stateowner->so_confirmed)
+	if (!open->op_stateowner->so_confirmed &&
+	    !nfsd4_has_session(&resp->cstate))
 		open->op_rflags |= NFS4_OPEN_RESULT_CONFIRM;
 
 	return status;
@@ -2797,8 +2808,15 @@ grace_disallows_io(struct inode *inode)
 	return locks_in_grace() && mandatory_lock(inode);
 }
 
-static int check_stateid_generation(stateid_t *in, stateid_t *ref)
+static int check_stateid_generation(stateid_t *in, stateid_t *ref, int flags)
 {
+	/*
+	 * When sessions are used the stateid generation number is ignored
+	 * when it is zero.
+	 */
+	if ((flags & HAS_SESSION) && in->si_generation == 0)
+		goto out;
+
 	/* If the client sends us a stateid from the future, it's buggy: */
 	if (in->si_generation > ref->si_generation)
 		return nfserr_bad_stateid;
@@ -2814,6 +2832,7 @@ static int check_stateid_generation(stateid_t *in, stateid_t *ref)
 	 */
 	if (in->si_generation < ref->si_generation)
 		return nfserr_old_stateid;
+out:
 	return nfs_ok;
 }
 
@@ -2851,7 +2870,8 @@ nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int fl
 		dp = find_delegation_stateid(ino, stateid);
 		if (!dp)
 			goto out;
-		status = check_stateid_generation(stateid, &dp->dl_stateid);
+		status = check_stateid_generation(stateid, &dp->dl_stateid,
+						  flags);
 		if (status)
 			goto out;
 		status = nfs4_check_delegmode(dp, flags);
@@ -2868,7 +2888,8 @@ nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int fl
 			goto out;
 		if (!stp->st_stateowner->so_confirmed)
 			goto out;
-		status = check_stateid_generation(stateid, &stp->st_stateid);
+		status = check_stateid_generation(stateid, &stp->st_stateid,
+						  flags);
 		if (status)
 			goto out;
 		status = nfs4_check_openmode(stp, flags);
@@ -2971,7 +2992,7 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
 	*  For the moment, we ignore the possibility of 
 	*  generation number wraparound.
 	*/
-	if (seqid != sop->so_seqid)
+	if (!(flags & HAS_SESSION) && seqid != sop->so_seqid)
 		goto check_replay;
 
 	if (sop->so_confirmed && flags & CONFIRM) {
@@ -2984,7 +3005,7 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
 				" confirmed yet!\n");
 		return nfserr_bad_stateid;
 	}
-	status = check_stateid_generation(stateid, &stp->st_stateid);
+	status = check_stateid_generation(stateid, &stp->st_stateid, flags);
 	if (status)
 		return status;
 	renew_client(sop->so_client);
@@ -3080,6 +3101,7 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
 	__be32 status;
 	struct nfs4_stateid *stp;
 	unsigned int share_access;
+	int flags = OPEN_STATE;
 
 	dprintk("NFSD: nfsd4_open_downgrade on file %.*s\n", 
 			(int)cstate->current_fh.fh_dentry->d_name.len,
@@ -3089,11 +3111,13 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
 			|| !deny_valid(od->od_share_deny))
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 					od->od_seqid,
 					&od->od_stateid, 
-					OPEN_STATE,
+					flags,
 					&od->od_stateowner, &stp, NULL)))
 		goto out; 
 
@@ -3136,17 +3160,20 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 {
 	__be32 status;
 	struct nfs4_stateid *stp;
+	int flags = OPEN_STATE | CLOSE_STATE;
 
 	dprintk("NFSD: nfsd4_close on file %.*s\n", 
 			(int)cstate->current_fh.fh_dentry->d_name.len,
 			cstate->current_fh.fh_dentry->d_name.name);
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	/* check close_lru for replay */
 	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 					close->cl_seqid,
 					&close->cl_stateid, 
-					OPEN_STATE | CLOSE_STATE,
+					flags,
 					&close->cl_stateowner, &stp, NULL)))
 		goto out; 
 	status = nfs_ok;
@@ -3179,11 +3206,14 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	stateid_t *stateid = &dr->dr_stateid;
 	struct inode *inode;
 	__be32 status;
+	int flags = 0;
 
 	if ((status = fh_verify(rqstp, &cstate->current_fh, S_IFREG, 0)))
 		return status;
 	inode = cstate->current_fh.fh_dentry->d_inode;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	status = nfserr_bad_stateid;
 	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
@@ -3197,7 +3227,7 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	dp = find_delegation_stateid(inode, stateid);
 	if (!dp)
 		goto out;
-	status = check_stateid_generation(stateid, &dp->dl_stateid);
+	status = check_stateid_generation(stateid, &dp->dl_stateid, flags);
 	if (status)
 		goto out;
 	renew_client(dp->dl_client);
@@ -3459,7 +3489,7 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	__be32 status = 0;
 	unsigned int strhashval;
 	unsigned int cmd;
-	int err;
+	int err, flags = 0;
 
 	dprintk("NFSD: nfsd4_lock: start=%Ld length=%Ld\n",
 		(long long) lock->lk_offset,
@@ -3489,11 +3519,15 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		if (STALE_CLIENTID(&lock->lk_new_clientid))
 			goto out;
 
+		flags = OPEN_STATE;
+		if (nfsd4_has_session(cstate))
+			flags |= HAS_SESSION;
+
 		/* validate and update open stateid and open seqid */
 		status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 				        lock->lk_new_open_seqid,
 		                        &lock->lk_new_open_stateid,
-					OPEN_STATE,
+					flags,
 		                        &lock->lk_replay_owner, &open_stp,
 					lock);
 		if (status)
@@ -3516,11 +3550,15 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		if (lock_stp == NULL)
 			goto out;
 	} else {
+		flags = LOCK_STATE;
+		if (nfsd4_has_session(cstate))
+			flags |= HAS_SESSION;
+
 		/* lock (lock owner + lock stateid) already exists */
 		status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 				       lock->lk_old_lock_seqid, 
 				       &lock->lk_old_lock_stateid, 
-				       LOCK_STATE,
+				       flags,
 				       &lock->lk_replay_owner, &lock_stp, lock);
 		if (status)
 			goto out;
@@ -3702,7 +3740,7 @@ nfsd4_locku(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	struct file *filp = NULL;
 	struct file_lock file_lock;
 	__be32 status;
-	int err;
+	int err, flags = LOCK_STATE;
 						        
 	dprintk("NFSD: nfsd4_locku: start=%Ld length=%Ld\n",
 		(long long) locku->lu_offset,
@@ -3711,12 +3749,14 @@ nfsd4_locku(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (check_lock_length(locku->lu_offset, locku->lu_length))
 		 return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 									        
 	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 					locku->lu_seqid, 
 					&locku->lu_stateid, 
-					LOCK_STATE,
+					flags,
 					&locku->lu_stateowner, &stp, NULL)))
 		goto out;
 
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 5720aab..a2682e8 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3206,7 +3206,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
 	BUG_ON(iov->iov_len > PAGE_SIZE);
 #ifdef CONFIG_NFSD_V4_1
-	if (resp->cstate.slot != NULL) {
+	if (nfsd4_has_session(&resp->cstate)) {
 		if (resp->cstate.status == nfserr_replay_cache &&
 				!nfsd4_no_page_in_cache(resp)) {
 			iov->iov_len = resp->cstate.iovlen;
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 47c7836..302557d 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -323,6 +323,7 @@ struct nfs4_stateid {
 };
 
 /* flags for preprocess_seqid_op() */
+#define HAS_SESSION             0x00000001
 #define CONFIRM                 0x00000002
 #define OPEN_STATE              0x00000004
 #define LOCK_STATE              0x00000008
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 37a7c51..aafbfdc 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -55,6 +55,11 @@ struct nfsd4_compound_state {
 	u32			status;
 };
 
+static inline bool nfsd4_has_session(struct nfsd4_compound_state *cs)
+{
+	return cs->slot != NULL;
+}
+
 struct nfsd4_change_info {
 	u32		atomic;
 	u32		before_ctime_sec;
@@ -540,7 +545,8 @@ extern __be32 nfsd4_destroy_session(struct svc_rqst *,
 		struct nfsd4_compound_state *,
 		struct nfsd4_destroy_session *);
 #endif /* CONFIG_NFSD_V4_1 */
-extern __be32 nfsd4_process_open1(struct nfsd4_open *open);
+extern __be32 nfsd4_process_open1(struct svc_rqst *rqstp,
+		struct nfsd4_open *open);
 extern __be32 nfsd4_process_open2(struct svc_rqst *rqstp,
 		struct svc_fh *current_fh, struct nfsd4_open *open);
 extern __be32 nfsd4_open_confirm(struct svc_rqst *rqstp,
-- 
1.6.2.1

[PATCH 29/47] nfsd41: check encode size for sessions maxresponse cached

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Calculate the space the compound response has taken after encoding the current
operation.

pad: add on 8 bytes for the next operation's op_code and status so that
there is room to cache a failure on the next operation.

Compare this length to the session se_fmaxresp_cached and return
nfserr_rep_too_big_to_cache if the length is too large.

Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
will be at least a page and will therefore hold the xdr_buf head.

Signed-off-by: Andy Adamson <andros@netapp.com>
[nfsd41: non-page DRC for solo sequence responses]
[fixed nfsd4_check_drc_limit cosmetics]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 58 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a2682e8..52ca833 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3089,6 +3089,61 @@ static nfsd4_enc nfsd4_enc_ops[] = {
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * Calculate the total amount of memory that the compound response has taken
+ * after encoding the current operation.
+ *
+ * pad: add on 8 bytes for the next operation's op_code and status so that
+ * there is room to cache a failure on the next operation.
+ *
+ * Compare this length to the session se_fmaxresp_cached.
+ *
+ * Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
+ * will be at least a page and will therefore hold the xdr_buf head.
+ */
+static int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
+{
+	int status = 0;
+	struct xdr_buf *xb = &resp->rqstp->rq_res;
+	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
+	struct nfsd4_session *session = NULL;
+	struct nfsd4_slot *slot = resp->cstate.slot;
+	u32 length, tlen = 0, pad = 8;
+
+	if (!nfsd4_has_session(&resp->cstate))
+		return status;
+
+	session = slot->sl_session;
+	if (session == NULL || slot->sl_cache_entry.ce_cachethis == 0)
+		return status;
+
+	if (resp->opcnt >= args->opcnt)
+		pad = 0; /* this is the last operation */
+
+	if (xb->page_len == 0) {
+		length = (char *)resp->p - (char *)xb->head[0].iov_base + pad;
+	} else {
+		if (xb->tail[0].iov_base && xb->tail[0].iov_len > 0)
+			tlen = (char *)resp->p - (char *)xb->tail[0].iov_base;
+
+		length = xb->head[0].iov_len + xb->page_len + tlen + pad;
+	}
+	dprintk("%s length %u, xb->page_len %u tlen %u pad %u\n", __func__,
+		length, xb->page_len, tlen, pad);
+
+	if (length <= session->se_fmaxresp_cached)
+		return status;
+	else
+		return nfserr_rep_too_big_to_cache;
+}
+#else /* CONFIG_NFSD_V4_1 */
+static inline int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
+{
+	return 0;
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
@@ -3105,6 +3160,9 @@ nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 	BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
 	       !nfsd4_enc_ops[op->opnum]);
 	op->status = nfsd4_enc_ops[op->opnum](resp, op->status, &op->u);
+	/* nfsd4_check_drc_limit guarantees enough room for error status */
+	if (!op->status && nfsd4_check_drc_limit(resp))
+		op->status = nfserr_rep_too_big_to_cache;
 status:
 	/*
 	 * Note: We write the status directly, instead of using WRITE32(),
-- 
1.6.2.1

[PATCH 30/47] nfsd41: clientid handling

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Extract the clientid from sessionid to set the op_clientid on open.
Verify that the clid for other stateful ops is zero for minorversion != 0
Do all other checks for stateful ops without sessions.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
[fixed whitespace indent]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c  |   12 ++++++++++++
 fs/nfsd/nfs4state.c |   14 ++++++++------
 fs/nfsd/nfs4xdr.c   |    9 +++++++++
 3 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 1d4b2b5..0076bb1 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -162,6 +162,15 @@ do_open_fhandle(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_
 	return status;
 }
 
+static void
+copy_clientid(clientid_t *clid, struct nfsd4_session *session)
+{
+	struct nfsd4_sessionid *sid =
+			(struct nfsd4_sessionid *)session->se_sessionid.data;
+
+	clid->cl_boot = sid->clientid.cl_boot;
+	clid->cl_id = sid->clientid.cl_id;
+}
 
 static __be32
 nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
@@ -176,6 +185,9 @@ nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (open->op_create && open->op_claim_type != NFS4_OPEN_CLAIM_NULL)
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		copy_clientid(&open->op_clientid, cstate->slot->sl_session);
+
 	nfs4_lock_state();
 
 	/* check seqid for replay. set nfs4_owner */
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index bf5b214..00ad1c2 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -616,8 +616,8 @@ STALE_CLIENTID(clientid_t *clid)
 {
 	if (clid->cl_boot == boot_time)
 		return 0;
-	dprintk("NFSD stale clientid (%08x/%08x)\n", 
-			clid->cl_boot, clid->cl_id);
+	dprintk("NFSD stale clientid (%08x/%08x) boot_time %08lx\n",
+		clid->cl_boot, clid->cl_id, boot_time);
 	return 1;
 }
 
@@ -2968,8 +2968,9 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
 		if (lock->lk_is_new) {
 			if (!sop->so_is_open_owner)
 				return nfserr_bad_stateid;
-			if (!same_clid(&clp->cl_clientid, lockclid))
-			       return nfserr_bad_stateid;
+			if (!(flags & HAS_SESSION) &&
+			    !same_clid(&clp->cl_clientid, lockclid))
+				return nfserr_bad_stateid;
 			/* stp is the open stateid */
 			status = nfs4_check_openmode(stp, lkflg);
 			if (status)
@@ -3516,7 +3517,8 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		struct nfs4_file *fp;
 		
 		status = nfserr_stale_clientid;
-		if (STALE_CLIENTID(&lock->lk_new_clientid))
+		if (!nfsd4_has_session(cstate) &&
+		    STALE_CLIENTID(&lock->lk_new_clientid))
 			goto out;
 
 		flags = OPEN_STATE;
@@ -3678,7 +3680,7 @@ nfsd4_lockt(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	nfs4_lock_state();
 
 	status = nfserr_stale_clientid;
-	if (STALE_CLIENTID(&lockt->lt_clientid))
+	if (!nfsd4_has_session(cstate) && STALE_CLIENTID(&lockt->lt_clientid))
 		goto out;
 
 	if ((status = fh_verify(rqstp, &cstate->current_fh, S_IFREG, 0))) {
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 52ca833..a1af3a3 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -189,6 +189,11 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	return p;
 }
 
+static int zero_clientid(clientid_t *clid)
+{
+	return (clid->cl_boot == 0) && (clid->cl_id == 0);
+}
+
 static int
 defer_free(struct nfsd4_compoundargs *argp,
 		void (*release)(const void *), void *p)
@@ -584,6 +589,8 @@ nfsd4_decode_lockt(struct nfsd4_compoundargs *argp, struct nfsd4_lockt *lockt)
 	READ_BUF(lockt->lt_owner.len);
 	READMEM(lockt->lt_owner.data, lockt->lt_owner.len);
 
+	if (argp->minorversion && !zero_clientid(&lockt->lt_clientid))
+		return nfserr_inval;
 	DECODE_TAIL;
 }
 
@@ -994,6 +1001,8 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
 	READ_BUF(rlockowner->rl_owner.len);
 	READMEM(rlockowner->rl_owner.data, rlockowner->rl_owner.len);
 
+	if (argp->minorversion && !zero_clientid(&rlockowner->rl_clientid))
+		return nfserr_inval;
 	DECODE_TAIL;
 }
 
-- 
1.6.2.1

[PATCH 31/47] nfsd41: access_valid

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

For nfs41, the open share flags are used also for
delegation "wants" and "signals".  Check that they are valid.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |    1 +
 fs/nfsd/nfs4state.c       |   20 +++++++++++++++-----
 include/linux/nfsd/xdr4.h |    1 +
 3 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 0076bb1..8b4bf5e 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -901,6 +901,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	resp->tag = args->tag;
 	resp->opcnt = 0;
 	resp->rqstp = rqstp;
+	resp->cstate.minorversion = args->minorversion;
 	resp->cstate.replay_owner = NULL;
 	fh_init(&resp->cstate.current_fh, NFS4_FHSIZE);
 	fh_init(&resp->cstate.save_fh, NFS4_FHSIZE);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 00ad1c2..dbf0d07 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1959,11 +1959,21 @@ find_file(struct inode *ino)
 	return NULL;
 }
 
-static inline int access_valid(u32 x)
+static inline int access_valid(u32 x, u32 minorversion)
 {
-	if (x < NFS4_SHARE_ACCESS_READ)
+	if ((x & NFS4_SHARE_ACCESS_MASK) < NFS4_SHARE_ACCESS_READ)
 		return 0;
-	if (x > NFS4_SHARE_ACCESS_BOTH)
+	if ((x & NFS4_SHARE_ACCESS_MASK) > NFS4_SHARE_ACCESS_BOTH)
+		return 0;
+	x &= ~NFS4_SHARE_ACCESS_MASK;
+	if (minorversion && x) {
+		if ((x & NFS4_SHARE_WANT_MASK) > NFS4_SHARE_WANT_CANCEL)
+			return 0;
+		if ((x & NFS4_SHARE_WHEN_MASK) > NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED)
+			return 0;
+		x &= ~(NFS4_SHARE_WANT_MASK | NFS4_SHARE_WHEN_MASK);
+	}
+	if (x)
 		return 0;
 	return 1;
 }
@@ -2511,7 +2521,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	__be32 status;
 
 	status = nfserr_inval;
-	if (!access_valid(open->op_share_access)
+	if (!access_valid(open->op_share_access, resp->cstate.minorversion)
 			|| !deny_valid(open->op_share_deny))
 		goto out;
 	/*
@@ -3108,7 +3118,7 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
 			(int)cstate->current_fh.fh_dentry->d_name.len,
 			cstate->current_fh.fh_dentry->d_name.name);
 
-	if (!access_valid(od->od_share_access)
+	if (!access_valid(od->od_share_access, cstate->minorversion)
 			|| !deny_valid(od->od_share_deny))
 		return nfserr_inval;
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index aafbfdc..881ff67 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -52,6 +52,7 @@ struct nfsd4_compound_state {
 	struct nfsd4_slot	*slot;
 	__be32			*statp;
 	size_t			iovlen;
+	u32			minorversion;
 	u32			status;
 };
 
-- 
1.6.2.1

[PATCH 32/47] nfsd41: add OPEN4_SHARE_ACCESS_WANT nfs4_stateid bmap

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Separate the access bits from the want bits and enable __set_bit to
work correctly with st_access_bmap.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index dbf0d07..8a64981 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1905,7 +1905,8 @@ init_stateid(struct nfs4_stateid *stp, struct nfs4_file *fp, struct nfsd4_open *
 	stp->st_stateid.si_generation = 0;
 	stp->st_access_bmap = 0;
 	stp->st_deny_bmap = 0;
-	__set_bit(open->op_share_access, &stp->st_access_bmap);
+	__set_bit(open->op_share_access & ~NFS4_SHARE_WANT_MASK,
+		  &stp->st_access_bmap);
 	__set_bit(open->op_share_deny, &stp->st_deny_bmap);
 	stp->st_openstp = NULL;
 }
-- 
1.6.2.1

[PATCH 33/47] nfsd41: provide support for minor version 1 at rpc level

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfsd/nfsd.h |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index 3bccec3..d0fe9b4 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -23,7 +23,11 @@
 /*
  * nfsd version
  */
+#if defined(CONFIG_NFSD_V4_1)
+#define NFSD_SUPPORTED_MINOR_VERSION	1
+#else /* CONFIG_NFSD_V4_1 */
 #define NFSD_SUPPORTED_MINOR_VERSION	0
+#endif /* CONFIG_NFSD_V4_1 */
 
 /*
  * Flags for nfsd_permission
-- 
1.6.2.1

[PATCH 34/47] nfsd: cleanup nfs4.0 callback encode routines

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Mimic the client and prepare to share the back channel xdr with NFSv4.1.
Bump the number of operations in each encode routine, then backfill the
number of operations.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   24 ++++++++++++++++--------
 1 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 290289b..7129b0c 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -140,8 +140,9 @@ struct nfs4_cb_compound_hdr {
 	int		status;
 	u32		ident;
 	u32		nops;
+	__be32		*nops_p;
 	u32		taglen;
-	char *		tag;
+	char		*tag;
 };
 
 static struct {
@@ -201,7 +202,7 @@ nfs_cb_stat_to_errno(int stat)
  * XDR encode
  */
 
-static int
+static void
 encode_cb_compound_hdr(struct xdr_stream *xdr, struct nfs4_cb_compound_hdr *hdr)
 {
 	__be32 * p;
@@ -210,12 +211,18 @@ encode_cb_compound_hdr(struct xdr_stream *xdr, struct nfs4_cb_compound_hdr *hdr)
 	WRITE32(0);            /* tag length is always 0 */
 	WRITE32(NFS4_MINOR_VERSION);
 	WRITE32(hdr->ident);
+	hdr->nops_p = p;
 	WRITE32(hdr->nops);
-	return 0;
 }
 
-static int
-encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec)
+static void encode_cb_nops(struct nfs4_cb_compound_hdr *hdr)
+{
+	*hdr->nops_p = htonl(hdr->nops);
+}
+
+static void
+encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
+		 struct nfs4_cb_compound_hdr *hdr)
 {
 	__be32 *p;
 	int len = cb_rec->cbr_fh.fh_size;
@@ -227,7 +234,7 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec)
 	WRITE32(cb_rec->cbr_trunc);
 	WRITE32(len);
 	WRITEMEM(&cb_rec->cbr_fh.fh_base, len);
-	return 0;
+	hdr->nops++;
 }
 
 static int
@@ -246,12 +253,13 @@ nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *a
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->cbr_ident,
-		.nops   = 1,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
-	return (encode_cb_recall(&xdr, args));
+	encode_cb_recall(&xdr, args, &hdr);
+	encode_cb_nops(&hdr);
+	return 0;
 }
 
 
-- 
1.6.2.1

[PATCH 35/47] nfsd: minorversion support for the back channel

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Prepare to share backchannel code with NFSv4.1.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c     |    7 ++++++-
 fs/nfsd/nfs4state.c        |    1 +
 include/linux/nfsd/state.h |    3 ++-
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 7129b0c..d8f2631 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -141,6 +141,7 @@ struct nfs4_cb_compound_hdr {
 	u32		ident;
 	u32		nops;
 	__be32		*nops_p;
+	u32		minorversion;
 	u32		taglen;
 	char		*tag;
 };
@@ -209,7 +210,7 @@ encode_cb_compound_hdr(struct xdr_stream *xdr, struct nfs4_cb_compound_hdr *hdr)
 
 	RESERVE_SPACE(16);
 	WRITE32(0);            /* tag length is always 0 */
-	WRITE32(NFS4_MINOR_VERSION);
+	WRITE32(hdr->minorversion);
 	WRITE32(hdr->ident);
 	hdr->nops_p = p;
 	WRITE32(hdr->nops);
@@ -251,8 +252,11 @@ static int
 nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_callback *cb =
+		(struct nfs4_callback *)req->rq_task->tk_client->cl_private;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->cbr_ident,
+		.minorversion = cb->cb_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
@@ -436,6 +440,7 @@ static int do_probe_callback(void *data)
 		goto out_release_client;
 
 	cb->cb_client = client;
+	client->cl_private = cb;
 	atomic_set(&cb->cb_set, 1);
 	put_nfs4_client(clp);
 	return 0;
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 8a64981..189f1c2 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -994,6 +994,7 @@ gen_callback(struct nfs4_client *clp, struct nfsd4_setclientid *se)
 	if ( !(parse_ipv4(se->se_callback_addr_len, se->se_callback_addr_val,
 	                 &cb->cb_addr, &cb->cb_port)))
 		goto out_err;
+	cb->cb_minorversion = 0;
 	cb->cb_prog = se->se_callback_prog;
 	cb->cb_ident = se->se_callback_ident;
 	return;
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 302557d..b78021a 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -93,7 +93,8 @@ struct nfs4_callback {
 	u32                     cb_addr;
 	unsigned short          cb_port;
 	u32                     cb_prog;
-	u32                     cb_ident;
+	u32			cb_minorversion;
+	u32                     cb_ident;	/* minorversion 0 only */
 	/* RPC client info */
 	atomic_t		cb_set;     /* successful CB_NULL call */
 	struct rpc_clnt *       cb_client;
-- 
1.6.2.1

[PATCH 36/47] nfsd41: sunrpc: Added rpc server-side backchannel handling

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

FIXME: bhalevy: write up commit message

Signed-off-by: Mike Sager <sager@netapp.com>
Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>

When the call direction is a reply, copy the xid and call direction into the
req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns
rpc_garbage.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/sunrpc/clnt.h    |    1 +
 include/linux/sunrpc/svcsock.h |    1 +
 include/linux/sunrpc/xprt.h    |    4 +
 net/sunrpc/clnt.c              |    1 +
 net/sunrpc/svcsock.c           |   81 +++++++++++
 net/sunrpc/xprt.c              |   41 ++++++-
 net/sunrpc/xprtsock.c          |  286 +++++++++++++++++++++++++++++++++++++++-
 7 files changed, 405 insertions(+), 10 deletions(-)

diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 928c547..0b32b00 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -111,6 +111,7 @@ struct rpc_create_args {
 	rpc_authflavor_t	authflavor;
 	unsigned long		flags;
 	char			*client_name;
+	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
 };
 
 /* Values for "flags" field */
diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h
index 483e103..e8fda10 100644
--- a/include/linux/sunrpc/svcsock.h
+++ b/include/linux/sunrpc/svcsock.h
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt	       *sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index 11fc71d..e9ffee3 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -174,6 +174,9 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+#if defined(CONFIG_NFSD_V4_1)
+	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
+#endif /* CONFIG_NFSD_V4_1 */
 	struct list_head	recv;
 
 	struct {
@@ -197,6 +200,7 @@ struct xprt_create {
 	struct sockaddr *	srcaddr;	/* optional local address */
 	struct sockaddr *	dstaddr;	/* remote peer address */
 	size_t			addrlen;
+	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
 };
 
 struct xprt_class {
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index 836f15c..0a8f2d0 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -266,6 +266,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
 		.srcaddr = args->saddress,
 		.dstaddr = args->address,
 		.addrlen = args->addrsize,
+		.bc_sock = args->bc_sock,
 	};
 	char servername[48];
 
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 7a2a90f..90b2232 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -49,6 +49,7 @@
 #include <linux/sunrpc/msg_prot.h>
 #include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/stats.h>
+#include <linux/sunrpc/xprt.h>
 
 #define RPCDBG_FACILITY	RPCDBG_SVCXPRT
 
@@ -790,6 +791,9 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	int		len;
 	struct kvec *vec;
 	int pnum, vlen;
+#if defined(CONFIG_NFSD_V4_1)
+	struct rpc_rqst *req = NULL;
+#endif
 
 	dprintk("svc: tcp_recv %p data %d conn %d close %d\n",
 		svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags),
@@ -856,12 +860,73 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	len = svsk->sk_reclen;
 	set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags);
 
+	/*
+	 * We have enough data for the whole tcp record. Let's try and read the
+	 * first 8 bytes to get the xid and the call direction. We can use this
+	 * to figure out if this is a call or a reply to a callback. If
+	 * sk_reclen is < 8 (xid and calldir), then this is a malformed packet.
+	 * In that case, don't bother with the calldir and just read the data.
+	 * It will be rejected in svc_process.
+	 */
+
 	vec = rqstp->rq_vec;
 	vec[0] = rqstp->rq_arg.head[0];
 	vlen = PAGE_SIZE;
+
+	if (len >= 8) {
+		u32 *p;
+		u32 xid;
+		u32 calldir;
+
+		len = svc_recvfrom(rqstp, vec, 1, 8);
+		if (len < 0)
+			goto error;
+
+		p = (u32 *)rqstp->rq_arg.head[0].iov_base;
+		xid = *p++;
+		calldir = *p;
+
+#if defined(CONFIG_NFSD_V4_1)
+		if (calldir) {
+			/* REPLY */
+			if (svsk->sk_bc_xprt)
+				req = xprt_lookup_rqst(svsk->sk_bc_xprt, xid);
+			if (req) {
+				memcpy(&req->rq_private_buf, &req->rq_rcv_buf,
+					sizeof(struct xdr_buf));
+				/* copy the xid and call direction */
+				memcpy(req->rq_private_buf.head[0].iov_base,
+					rqstp->rq_arg.head[0].iov_base, 8);
+				vec[0] = req->rq_private_buf.head[0];
+			} else
+				printk(KERN_NOTICE
+					"%s: Got unrecognized reply: "
+					"calldir 0x%x sk_bc_xprt %p xid %08x\n",
+					__func__, ntohl(calldir),
+					svsk->sk_bc_xprt, xid);
+		}
+
+		if (!calldir || !req)
+			vec[0] = rqstp->rq_arg.head[0];
+
+#else /* CONFIG_NFSD_V4_1 */
+		vec[0] = rqstp->rq_arg.head[0];
+#endif /* CONFIG_NFSD_V4_1 */
+		vec[0].iov_base += 8;
+		vec[0].iov_len -= 8;
+		len = svsk->sk_reclen - 8;
+		vlen -= 8;
+	}
+
 	pnum = 1;
 	while (vlen < len) {
+#if defined(CONFIG_NFSD_V4_1)
+		vec[pnum].iov_base = (req) ?
+			page_address(req->rq_private_buf.pages[pnum - 1]) :
+			page_address(rqstp->rq_pages[pnum]);
+#else /* CONFIG_NFSD_V4_1 */
 		vec[pnum].iov_base = page_address(rqstp->rq_pages[pnum]);
+#endif /* CONFIG_NFSD_V4_1 */
 		vec[pnum].iov_len = PAGE_SIZE;
 		pnum++;
 		vlen += PAGE_SIZE;
@@ -873,6 +938,18 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	if (len < 0)
 		goto error;
 
+	/*
+	 * Account for the 8 bytes we read earlier
+	 */
+	len += 8;
+
+#if defined(CONFIG_NFSD_V4_1)
+	if (req) {
+		xprt_complete_rqst(req->rq_task, len);
+		len = 0;
+		goto out;
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 	dprintk("svc: TCP complete record (%d bytes)\n", len);
 	rqstp->rq_arg.len = len;
 	rqstp->rq_arg.page_base = 0;
@@ -886,6 +963,10 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	rqstp->rq_xprt_ctxt   = NULL;
 	rqstp->rq_prot	      = IPPROTO_TCP;
 
+#if defined(CONFIG_NFSD_V4_1)
+out:
+#endif /* CONFIG_NFSD_V4_1 */
+
 	/* Reset TCP read info */
 	svsk->sk_reclen = 0;
 	svsk->sk_tcplen = 0;
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index 62098d1..7fac90c 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -998,6 +998,27 @@ void xprt_release(struct rpc_task *task)
 	spin_unlock(&xprt->reserve_lock);
 }
 
+/*
+ * The autoclose function for the back channel
+ *
+ * The callback channel should never close the channel,
+ * let the forechannel do that.
+ */
+static void bc_autoclose(struct work_struct *work)
+{
+	return;
+}
+
+
+/*
+ * The autodisconnect routine for the back channel. We never disconnect
+ */
+static void
+bc_init_autodisconnect(unsigned long data)
+{
+	return;
+}
+
 /**
  * xprt_create_transport - create an RPC transport
  * @args: rpc transport creation arguments
@@ -1034,9 +1055,16 @@ found:
 
 	INIT_LIST_HEAD(&xprt->free);
 	INIT_LIST_HEAD(&xprt->recv);
-	INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
-	setup_timer(&xprt->timer, xprt_init_autodisconnect,
-			(unsigned long)xprt);
+	if (args->bc_sock) {
+		INIT_WORK(&xprt->task_cleanup, bc_autoclose);
+		setup_timer(&xprt->timer, bc_init_autodisconnect,
+			    (unsigned long)xprt);
+	} else {
+		INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
+		setup_timer(&xprt->timer, xprt_init_autodisconnect,
+			    (unsigned long)xprt);
+	}
+
 	xprt->last_used = jiffies;
 	xprt->cwnd = RPC_INITCWND;
 	xprt->bind_index = 0;
@@ -1056,6 +1084,13 @@ found:
 	dprintk("RPC:       created transport %p with %u slots\n", xprt,
 			xprt->max_reqs);
 
+	/*
+	 * Since we don't want connections for the backchannel, we set
+	 * the xprt status to connected
+	 */
+	if (args->bc_sock)
+		xprt_set_connected(xprt);
+
 	return xprt;
 }
 
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 29c71e6..f90c7c3 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -32,6 +32,7 @@
 #include <linux/tcp.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/sched.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/xprtsock.h>
 #include <linux/file.h>
 
@@ -1884,6 +1885,221 @@ static void xs_tcp_print_stats(struct rpc_xprt *xprt, struct seq_file *seq)
 			xprt->stat.bklog_u);
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * The connect worker for the backchannel
+ * This should never be called as we should never need to connect
+ */
+static void bc_connect_worker(struct work_struct *work)
+{
+	BUG();
+}
+
+/*
+ * The set_port routine of the rpc_xprt_ops. This is related to the portmapper
+ * and should never be called
+ */
+
+static void bc_set_port(struct rpc_xprt *xprt, unsigned short port)
+{
+	BUG();
+}
+
+/*
+ * The connect routine for the backchannel rpc_xprt ops
+ * Again, should never be called!
+ */
+
+static void bc_connect(struct rpc_task *task)
+{
+	BUG();
+}
+
+struct rpc_buffer {
+	size_t	len;
+	char	data[];
+};
+/*
+ * Allocate a bunch of pages for a scratch buffer for the rpc code. The reason
+ * we allocate pages instead doing a kmalloc like rpc_malloc is because we want
+ * to use the server side send routines.
+ */
+void *bc_malloc(struct rpc_task *task, size_t size)
+{
+	struct page *page;
+	struct rpc_buffer *buf;
+
+	BUG_ON(size > PAGE_SIZE - sizeof(struct rpc_buffer));
+	page = alloc_page(GFP_KERNEL);
+
+	if (!page)
+		return NULL;
+
+	buf = page_address(page);
+	buf->len = PAGE_SIZE;
+
+	return buf->data;
+}
+
+/*
+ * Free the space allocated in the bc_alloc routine
+ */
+void bc_free(void *buffer)
+{
+	struct rpc_buffer *buf;
+
+	if (!buffer)
+		return;
+
+	buf = container_of(buffer, struct rpc_buffer, data);
+	free_pages((unsigned long)buf, get_order(buf->len));
+}
+
+/*
+ * Use the svc_sock to send the callback. Must be called with svsk->sk_mutex
+ * held. Borrows heavily from svc_tcp_sendto and xs_tcp_semd_request.
+ */
+static int bc_sendto(struct rpc_rqst *req)
+{
+	int total_len;
+	int len;
+	int size;
+	int result;
+	struct xdr_buf *xbufp = &req->rq_snd_buf;
+	struct page **pages = xbufp->pages;
+	unsigned int flags = MSG_MORE;
+	unsigned int pglen = xbufp->page_len;
+	size_t base = xbufp->page_base;
+	struct rpc_xprt *xprt = req->rq_xprt;
+	struct sock_xprt *transport =
+				container_of(xprt, struct sock_xprt, xprt);
+	struct socket *sock = transport->sock;
+
+	total_len = xbufp->len;
+
+	/*
+	 * Set up the rpc header and record marker stuff
+	 */
+	xs_encode_tcp_record_marker(xbufp);
+
+	/*
+	 * The RPC message is divided into 3 pieces:
+	 * - The header: This is what most of the smaller RPC messages consist
+	 *   of. Often the whole message is in this.
+	 *
+	 *   - xdr->pages: This is a list of pages that contain data, for
+	 *   example in a write request or while using rpcsec gss
+	 *
+	 *   - The tail: This is the rest of the rpc message
+	 *
+	 *  First we send the header, then the pages and then finally the tail.
+	 *  The code borrows heavily from svc_sendto.
+	 */
+
+	/*
+	 * Send the head
+	 */
+	if (total_len == xbufp->head[0].iov_len)
+		flags = 0;
+
+	len = sock->ops->sendpage(sock, virt_to_page(xbufp->head[0].iov_base),
+			(unsigned long)xbufp->head[0].iov_base & ~PAGE_MASK,
+			xbufp->head[0].iov_len, flags);
+
+	if (len != xbufp->head[0].iov_len)
+		goto out;
+
+	/*
+	 * send page data
+	 *
+	 * Check the amount of data to be sent. If it is less than the
+	 * remaining page, then send it else send the current page
+	 */
+
+	size = PAGE_SIZE - base < pglen ? PAGE_SIZE - base : pglen;
+	while (pglen > 0) {
+		if (total_len == size)
+			flags = 0;
+		result = sock->ops->sendpage(sock, *pages, base, size, flags);
+		if (result > 0)
+			len += result;
+		if (result != size)
+			goto out;
+		total_len -= size;
+		pglen -= size;
+		size = PAGE_SIZE < pglen ? PAGE_SIZE : pglen;
+		base = 0;
+		pages++;
+	}
+	/*
+	 * send tail
+	 */
+	if (xbufp->tail[0].iov_len) {
+		result = sock->ops->sendpage(sock,
+			xbufp->tail[0].iov_base,
+			(unsigned long)xbufp->tail[0].iov_base & ~PAGE_MASK,
+			xbufp->tail[0].iov_len,
+			0);
+
+		if (result > 0)
+			len += result;
+	}
+out:
+	if (len != xbufp->len)
+		printk(KERN_NOTICE "Error sending entire callback!\n");
+
+	return len;
+}
+
+/*
+ * The send routine. Borrows from svc_send
+ */
+static int bc_send_request(struct rpc_task *task)
+{
+	struct rpc_rqst *req = task->tk_rqstp;
+	struct rpc_xprt *bc_xprt = req->rq_xprt;
+	struct svc_xprt	*xprt;
+	struct svc_sock         *svsk;
+	u32                     len;
+
+	dprintk("sending request with xid: %08x\n", ntohl(req->rq_xid));
+	/*
+	 * Get the server socket associated with this callback xprt
+	 */
+	svsk = bc_xprt->bc_sock;
+	xprt = &svsk->sk_xprt;
+
+	mutex_lock(&xprt->xpt_mutex);
+	if (test_bit(XPT_DEAD, &xprt->xpt_flags))
+		len = -ENOTCONN;
+	else
+		len = bc_sendto(req);
+	mutex_unlock(&xprt->xpt_mutex);
+
+	return 0;
+
+}
+
+/*
+ * The close routine. Since this is client initiated, we do nothing
+ */
+
+static void bc_close(struct rpc_xprt *xprt)
+{
+	return;
+}
+
+/*
+ * The xprt destroy routine. Again, because this connection is client
+ * initiated, we do nothing
+ */
+
+static void bc_destroy(struct rpc_xprt *xprt)
+{
+	return;
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static struct rpc_xprt_ops xs_udp_ops = {
 	.set_buffer_size	= xs_udp_set_buffer_size,
 	.reserve_xprt		= xprt_reserve_xprt_cong,
@@ -1917,6 +2133,26 @@ static struct rpc_xprt_ops xs_tcp_ops = {
 	.print_stats		= xs_tcp_print_stats,
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * The rpc_xprt_ops for the server backchannel
+ */
+
+static struct rpc_xprt_ops bc_tcp_ops = {
+	.reserve_xprt		= xprt_reserve_xprt,
+	.release_xprt		= xprt_release_xprt,
+	.set_port		= bc_set_port,
+	.connect		= bc_connect,
+	.buf_alloc		= bc_malloc,
+	.buf_free		= bc_free,
+	.send_request		= bc_send_request,
+	.set_retrans_timeout	= xprt_set_retrans_timeout_def,
+	.close			= bc_close,
+	.destroy		= bc_destroy,
+	.print_stats		= xs_tcp_print_stats,
+};
+#endif /* CONFIG_NFSD_V4_1 */
+
 static struct rpc_xprt *xs_setup_xprt(struct xprt_create *args,
 				      unsigned int slot_table_size)
 {
@@ -2049,13 +2285,31 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
 	xprt->tsh_size = sizeof(rpc_fraghdr) / sizeof(u32);
 	xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
 
-	xprt->bind_timeout = XS_BIND_TO;
-	xprt->connect_timeout = XS_TCP_CONN_TO;
-	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
-	xprt->idle_timeout = XS_IDLE_DISC_TO;
+#ifdef CONFIG_NFSD_V4_1
+	if (args->bc_sock) {
+		/* backchannel */
+		xprt_set_bound(xprt);
+		INIT_DELAYED_WORK(&transport->connect_worker,
+				  bc_connect_worker);
+		xprt->bind_timeout = 0;
+		xprt->connect_timeout = 0;
+		xprt->reestablish_timeout = 0;
+		xprt->idle_timeout = (~0);
 
-	xprt->ops = &xs_tcp_ops;
-	xprt->timeout = &xs_tcp_default_timeout;
+		/*
+		 * The backchannel uses the same socket connection as the
+		 * forechannel
+		 */
+		xprt->bc_sock = args->bc_sock;
+		xprt->bc_sock->sk_bc_xprt = xprt;
+		transport->sock = xprt->bc_sock->sk_sock;
+		transport->inet = xprt->bc_sock->sk_sk;
+
+		xprt->ops = &bc_tcp_ops;
+
+		goto next;
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 
 	switch (addr->sa_family) {
 	case AF_INET:
@@ -2063,13 +2317,31 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
 			xprt_set_bound(xprt);
 
 		INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_connect_worker4);
-		xs_format_ipv4_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP);
 		break;
 	case AF_INET6:
 		if (((struct sockaddr_in6 *)addr)->sin6_port != htons(0))
 			xprt_set_bound(xprt);
 
 		INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_connect_worker6);
+		break;
+	}
+	xprt->bind_timeout = XS_BIND_TO;
+	xprt->connect_timeout = XS_TCP_CONN_TO;
+	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
+	xprt->idle_timeout = XS_IDLE_DISC_TO;
+
+	xprt->ops = &xs_tcp_ops;
+
+#ifdef CONFIG_NFSD_V4_1
+next:
+#endif /* CONFIG_NFSD_V4_1 */
+	xprt->timeout = &xs_tcp_default_timeout;
+
+	switch (addr->sa_family) {
+	case AF_INET:
+		xs_format_ipv4_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP);
+		break;
+	case AF_INET6:
 		xs_format_ipv6_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP6);
 		break;
 	default:
-- 
1.6.2.1

[PATCH 37/47] nfsd41: callback infrastructure

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Keep the xprt used for create_session in cl_cb_xprt.
Mark cl_callback.cb_minorversion = 1 and remember
the client provided cl_callback.cb_prog rpc program number.
Use it to probe the callback path.

Define xdr sizes and code nfs4_cb_compound header to be able
to send a null callback rpc.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   35 ++++++++++++++++++++++++++++++++---
 fs/nfsd/nfs4state.c    |   10 ++++++++++
 2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index d8f2631..6b7ef87 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
-        NFSPROC4_CLNT_CB_NULL = 0,
+	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -135,13 +146,19 @@ xdr_error:                                      \
 		return -EIO; \
 	} \
 } while (0)
+#define COPYMEM(x, nbytes) do {                \
+	memcpy((x), p, nbytes);                \
+	p += XDR_QUADLEN(nbytes);              \
+} while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -405,6 +422,17 @@ static struct rpc_clnt *setup_callback_client(struct nfs4_client *clp)
 	addr.sin_family = AF_INET;
 	addr.sin_port = htons(cb->cb_port);
 	addr.sin_addr.s_addr = htonl(cb->cb_addr);
+#if defined(CONFIG_NFSD_V4_1)
+	if (cb->cb_minorversion) {
+		BUG_ON(cb->cb_minorversion != 1);
+		args.bc_sock = container_of(clp->cl_cb_xprt, struct svc_sock,
+					    sk_xprt);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
+
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
 
 	/* Create RPC client */
 	client = rpc_create(&args);
@@ -445,6 +473,7 @@ static int do_probe_callback(void *data)
 	put_nfs4_client(clp);
 	return 0;
 out_release_client:
+	dprintk("NFSD: synchronous CB_NULL failed. status=%d\n", status);
 	rpc_shutdown_client(client);
 out_err:
 	dprintk("NFSD: warning: no callback path to client %.*s: error %d\n",
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 189f1c2..ef1e372 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -663,6 +663,8 @@ free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
 #if defined(CONFIG_NFSD_V4_1)
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	nfsd4_release_respages(clp->cl_slot.sl_cache_entry.ce_respages,
 			     clp->cl_slot.sl_cache_entry.ce_resused);
 #endif /* CONFIG_NFSD_V4_1 */
@@ -1411,6 +1413,14 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			unconf->cl_callback.cb_minorversion = 1;
+			unconf->cl_cb_seq_nr = 0;
+			unconf->cl_callback.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
-- 
1.6.2.1

[PATCH 38/47] nfsd41: Remember the auth flavor to use for callbacks

[Benny Halevy]

From: Ricardo Labiaga <Ricardo.Labiaga@netapp.com>

The callbacks will be sent using the same authentication flavor that
was used during session creation.  We'll add code to remember the
principal in the case RPCSEC_GSS in a separate patch.

Signed-off-by: Ricardo Labiaga <Ricardo.Labiaga@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index ef1e372..eb3d0dc 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1307,6 +1307,7 @@ out_new:
 	copy_verf(new, &verf);
 	copy_cred(&new->cl_cred, &rqstp->rq_cred);
 	new->cl_addr = ip_addr;
+	new->cl_flavor = rqstp->rq_flavor;
 	gen_clid(new);
 	gen_confirm(new);
 	add_to_unconfirmed(new, strhashval);
-- 
1.6.2.1

[PATCH 39/47] nfsd41: introduce cl_cb_mutex

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

To be used for serializing nfs41 callbacks.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |    1 +
 include/linux/nfsd/state.h |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index eb3d0dc..e5c7ddb 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -741,6 +741,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
 	INIT_LIST_HEAD(&clp->cl_delegations);
 #if defined(CONFIG_NFSD_V4_1)
 	INIT_LIST_HEAD(&clp->cl_sessions);
+	mutex_init(&clp->cl_cb_mutex);
 #endif /* CONFIG_NFSD_V4_1 */
 	INIT_LIST_HEAD(&clp->cl_lru);
 	return clp;
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index b78021a..f2783d4 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -197,6 +197,7 @@ struct nfs4_client {
 	struct nfs4_sessionid	cl_sessionid;
 
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
+	struct mutex		cl_cb_mutex;
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
-- 
1.6.2.1

[PATCH 40/47] nfsd41: cb_sequence callback

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
Signed-off-by: Andy Adamson <andros@netapp.com>
[fixed indentation]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfsd/state.h |    7 ++-
 2 files changed, 124 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 6b7ef87..7ada6b1 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
 	hdr->nops++;
 }
 
+static void
+encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
+		   struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	if (hdr->minorversion == 0)
+		return;
+
+	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
+
+	WRITE32(OP_CB_SEQUENCE);
+#ifdef CONFIG_NFSD_V4_1
+	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(args->cbs_clp->cl_cb_seq_nr);
+#endif /* CONFIG_NFSD_V4_1 */
+	WRITE32(0);		/* slotid, always 0 */
+	WRITE32(0);		/* highest slotid always 0 */
+	WRITE32(0);		/* cachethis always 0 */
+	WRITE32(0); /* FIXME: support referring_call_lists */
+	hdr->nops++;
+}
+
 static int
 nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
 	return 0;
 }
 
+/*
+ * Our current back channel implmentation supports a single backchannel
+ * with a single slot.
+ */
+static int
+decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
+		   struct rpc_rqst *rqstp)
+{
+	struct nfs4_callback *cb =
+		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
+	struct nfs4_sessionid id;
+	int status;
+	u32 dummy;
+	__be32 *p;
+
+	if (cb->cb_minorversion == 0)
+		return 0;
+
+	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
+	if (status)
+		return status;
+
+	/*
+	 * If the server returns different values for sessionID, slotID or
+	 * sequence number, the server is looney tunes.
+	 */
+	status = -ESERVERFAULT;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
+#ifdef CONFIG_NFSD_V4_1
+	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
+		   NFS4_MAX_SESSIONID_LEN)) {
+		dprintk("%s Invalid session id\n", __func__);
+		goto out;
+	}
+	READ32(dummy);
+	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
+		dprintk("%s Invalid sequence number\n", __func__);
+		goto out;
+	}
+#endif /* CONFIG_NFSD_V4_1 */
+	READ32(dummy); 	/* slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid slotid\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* highest slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid highest slotid\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* target highest slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid target highest slotid\n", __func__);
+		goto out;
+	}
+	status = 0;
+out:
+	return status;
+}
+
+
 static int
 nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
 	return;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * FIXME: cb_sequence should support referring call lists, cachethis, and
+ * multiple slots
+ */
+static int
+nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
+{
+	u32 *ptr = (u32 *)clp->cl_sessionid.data;
+
+	dprintk("%s: %u:%u:%u:%u\n", __func__,
+		ptr[0], ptr[1], ptr[2], ptr[3]);
+
+	mutex_lock(&clp->cl_cb_mutex);
+	args->cbs_clp = clp;
+	clp->cl_cb_seq_nr++;
+	return 0;
+}
+
+static void
+nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
+{
+	u32 *ptr = (u32 *)clp->cl_sessionid.data;
+
+	dprintk("%s: %u:%u:%u:%u\n", __func__,
+		ptr[0], ptr[1], ptr[2], ptr[3]);
+
+	/* FIXME: support multiple callback slots */
+	mutex_unlock(&clp->cl_cb_mutex);
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 /*
  * called with dp->dl_count inc'ed.
  */
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index f2783d4..0ea8c89 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -61,6 +61,10 @@ typedef struct {
 #define si_stateownerid   si_opaque.so_stateownerid
 #define si_fileid         si_opaque.so_fileid
 
+struct nfsd4_cb_sequence {
+	/* args/res */
+	struct nfs4_client	*cbs_clp;
+};
 
 struct nfs4_cb_recall {
 	u32			cbr_ident;
@@ -195,7 +199,8 @@ struct nfs4_client {
 	struct nfsd4_slot	cl_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
-
+	/* We currently support a single back channel with a single slot */
+	u32			cl_cb_seq_nr;
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 	struct mutex		cl_cb_mutex;
 #endif /* CONFIG_NFSD_V4_1 */
-- 
1.6.2.1

[PATCH 41/47] nfsd41: introduce nfs4_cb_call_sync for nfs4 and nfs41

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

For nfs41 callbacks we need to setup the cb_sequence args and
process the result.  This will be called by code common to
nfs4 and nfs41.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   50 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 50 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 7ada6b1..dd3b739 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -92,6 +92,15 @@ enum nfs_cb_opnum4 {
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
+struct nfs4_rpc_args {
+	void                     *args_op;
+	struct nfsd4_cb_sequence *args_seq;
+};
+
+struct nfs4_rpc_res {
+	struct nfsd4_cb_sequence *res_seq;
+};
+
 /*
 * Generic encode routines from fs/nfs/nfs4xdr.c
 */
@@ -589,6 +598,12 @@ nfsd4_probe_callback(struct nfs4_client *clp)
 	return;
 }
 
+static int _nfsd4_cb_sync(struct nfs4_client *clp,
+			  const struct rpc_message *msg, int flags)
+{
+	return rpc_call_sync(clp->cl_callback.cb_client, msg, RPC_TASK_SOFT);
+}
+
 #if defined(CONFIG_NFSD_V4_1)
 /*
  * FIXME: cb_sequence should support referring call lists, cachethis, and
@@ -619,6 +634,41 @@ nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
 	/* FIXME: support multiple callback slots */
 	mutex_unlock(&clp->cl_cb_mutex);
 }
+
+static int _nfsd41_cb_sync(struct nfs4_client *clp,
+			   struct rpc_message *msg, int flags)
+{
+	struct nfsd4_cb_sequence seq;
+	struct nfs4_rpc_args *args;
+	struct nfs4_rpc_res res;
+	int status;
+
+	args = msg->rpc_argp;
+	args->args_seq = &seq;
+
+	res.res_seq = &seq;
+	msg->rpc_resp = &res;
+
+	nfs41_cb_sequence_setup(clp, &seq);
+	status = _nfsd4_cb_sync(clp, msg, flags);
+	nfs41_cb_sequence_done(clp, &seq);
+
+	return status;
+}
+
+static int nfsd4_cb_sync(struct nfs4_client *clp,
+			 struct rpc_message *msg, int flags)
+{
+	return clp->cl_callback.cb_minorversion ?
+		_nfsd41_cb_sync(clp, msg, flags) :
+		_nfsd4_cb_sync(clp, msg, flags);
+}
+#else  /* CONFIG_NFSD_V4_1 */
+static int nfsd4_cb_sync(struct nfs4_client *clp,
+			 struct rpc_message *msg, int flags)
+{
+	return _nfsd4_cb_sync(clp, msg, flags);
+}
 #endif /* CONFIG_NFSD_V4_1 */
 
 /*
-- 
1.6.2.1

[PATCH 42/47] nfsd41: cb_recall callback

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the cb_recall callback conforming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

Signed-off-by: Ricardo Labiaga <ricardo.labiaga@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[Share v4.0 and v4.1 back channel xdr]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   25 +++++++++++++++++++------
 1 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index dd3b739..0be75e0 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -298,9 +298,11 @@ nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *args)
+nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p,
+		       struct nfs4_rpc_args *rpc_args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_cb_recall *args = rpc_args->args_op;
 	struct nfs4_callback *cb =
 		(struct nfs4_callback *)req->rq_task->tk_client->cl_private;
 	struct nfs4_cb_compound_hdr hdr = {
@@ -310,6 +312,7 @@ nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *a
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
+	encode_cb_sequence(&xdr, rpc_args->args_seq, &hdr);
 	encode_cb_recall(&xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 	return 0;
@@ -421,7 +424,8 @@ nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
+nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
+		       struct nfs4_rpc_res *rpc_res)
 {
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr;
@@ -431,6 +435,9 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
+	status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
+	if (status)
+		goto out;
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;
@@ -678,19 +685,23 @@ void
 nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
-	struct rpc_clnt *clnt = clp->cl_callback.cb_client;
 	struct nfs4_cb_recall *cbr = &dp->dl_recall;
+	struct nfs4_rpc_args args = {
+		.args_op = cbr,
+	};
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = cbr,
+		.rpc_argp = &args,
 	};
 	int retries = 1;
 	int status = 0;
 
+	dprintk("%s: dp %p\n", __func__, dp);
+
 	cbr->cbr_trunc = 0; /* XXX need to implement truncate optimization */
 	cbr->cbr_dp = dp;
 
-	status = rpc_call_sync(clnt, &msg, RPC_TASK_SOFT);
+	status = nfsd4_cb_sync(clp, &msg, RPC_TASK_SOFT);
 	while (retries--) {
 		switch (status) {
 			case -EIO:
@@ -705,13 +716,15 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 				goto out_put_cred;
 		}
 		ssleep(2);
-		status = rpc_call_sync(clnt, &msg, RPC_TASK_SOFT);
+		status = nfsd4_cb_sync(clp, &msg, RPC_TASK_SOFT);
 	}
 out_put_cred:
 	/*
 	 * Success or failure, now we're either waiting for lease expiration
 	 * or deleg_return.
 	 */
+	dprintk("%s: dp %p dl_flock %p dl_count %d\n",
+		__func__, dp, dp->dl_flock, atomic_read(&dp->dl_count));
 	put_nfs4_client(clp);
 	nfs4_put_delegation(dp);
 	return;
-- 
1.6.2.1

[PATCH 43/47] nfsd41: pass writable attrs mask to nfsd4_decode_fattr

[Benny Halevy]

In preparation for EXCLUSIVE4_1

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c |   21 +++++++++++++++------
 1 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a1af3a3..96f3eeb 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -251,9 +251,14 @@ nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 	DECODE_TAIL;
 }
 
+static u32 nfsd_attrmask[] = {
+	NFSD_WRITEABLE_ATTRS_WORD0,
+	NFSD_WRITEABLE_ATTRS_WORD1
+};
+
 static __be32
-nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, struct iattr *iattr,
-    struct nfs4_acl **acl)
+nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
+		   struct iattr *iattr, struct nfs4_acl **acl)
 {
 	int expected_len, len = 0;
 	u32 dummy32;
@@ -271,7 +276,7 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, struct iattr *ia
 	 */
 	if ((bmval[0] & ~NFSD_SUPPORTED_ATTRS_WORD0) || (bmval[1] & ~NFSD_SUPPORTED_ATTRS_WORD1))
 		return nfserr_attrnotsupp;
-	if ((bmval[0] & ~NFSD_WRITEABLE_ATTRS_WORD0) || (bmval[1] & ~NFSD_WRITEABLE_ATTRS_WORD1))
+	if ((bmval[0] & ~writable[0]) || (bmval[1] & ~writable[1]))
 		return nfserr_inval;
 
 	READ_BUF(4);
@@ -499,7 +504,9 @@ nfsd4_decode_create(struct nfsd4_compoundargs *argp, struct nfsd4_create *create
 	if ((status = check_filename(create->cr_name, create->cr_namelen, nfserr_inval)))
 		return status;
 
-	if ((status = nfsd4_decode_fattr(argp, create->cr_bmval, &create->cr_iattr, &create->cr_acl)))
+	status = nfsd4_decode_fattr(argp, create->cr_bmval, nfsd_attrmask,
+				    &create->cr_iattr, &create->cr_acl);
+	if (status)
 		goto out;
 
 	DECODE_TAIL;
@@ -660,7 +667,9 @@ nfsd4_decode_open(struct nfsd4_compoundargs *argp, struct nfsd4_open *open)
 		switch (open->op_createmode) {
 		case NFS4_CREATE_UNCHECKED:
 		case NFS4_CREATE_GUARDED:
-			if ((status = nfsd4_decode_fattr(argp, open->op_bmval, &open->op_iattr, &open->op_acl)))
+			status = nfsd4_decode_fattr(argp, open->op_bmval,
+				nfsd_attrmask, &open->op_iattr, &open->op_acl);
+			if (status)
 				goto out;
 			break;
 		case NFS4_CREATE_EXCLUSIVE:
@@ -859,7 +868,7 @@ nfsd4_decode_setattr(struct nfsd4_compoundargs *argp, struct nfsd4_setattr *seta
 	status = nfsd4_decode_stateid(argp, &setattr->sa_stateid);
 	if (status)
 		return status;
-	return nfsd4_decode_fattr(argp, setattr->sa_bmval,
+	return nfsd4_decode_fattr(argp, setattr->sa_bmval, nfsd_attrmask,
 				  &setattr->sa_iattr, &setattr->sa_acl);
 }
 
-- 
1.6.2.1

[PATCH 44/47] nfsd41: support for 3-word long attribute bitmask

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Also, use client minorversion to generate supported attrs

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   15 +++++++----
 fs/nfsd/nfs4xdr.c         |   59 +++++++++++++++++++++++++++++++++++---------
 include/linux/nfsd/nfsd.h |   48 +++++++++++++++++++++++++++++++++++-
 include/linux/nfsd/xdr4.h |   12 ++++----
 4 files changed, 108 insertions(+), 26 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 8b4bf5e..f3b560f 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -460,8 +460,9 @@ nfsd4_getattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (getattr->ga_bmval[1] & NFSD_WRITEONLY_ATTRS_WORD1)
 		return nfserr_inval;
 
-	getattr->ga_bmval[0] &= NFSD_SUPPORTED_ATTRS_WORD0;
-	getattr->ga_bmval[1] &= NFSD_SUPPORTED_ATTRS_WORD1;
+	getattr->ga_bmval[0] &= nfsd_suppattrs0(cstate->minorversion);
+	getattr->ga_bmval[1] &= nfsd_suppattrs1(cstate->minorversion);
+	getattr->ga_bmval[2] &= nfsd_suppattrs2(cstate->minorversion);
 
 	getattr->ga_fhp = &cstate->current_fh;
 	return nfs_ok;
@@ -556,8 +557,9 @@ nfsd4_readdir(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (readdir->rd_bmval[1] & NFSD_WRITEONLY_ATTRS_WORD1)
 		return nfserr_inval;
 
-	readdir->rd_bmval[0] &= NFSD_SUPPORTED_ATTRS_WORD0;
-	readdir->rd_bmval[1] &= NFSD_SUPPORTED_ATTRS_WORD1;
+	readdir->rd_bmval[0] &= nfsd_suppattrs0(cstate->minorversion);
+	readdir->rd_bmval[1] &= nfsd_suppattrs1(cstate->minorversion);
+	readdir->rd_bmval[2] &= nfsd_suppattrs2(cstate->minorversion);
 
 	if ((cookie > ~(u32)0) || (cookie == 1) || (cookie == 2) ||
 	    (cookie == 0 && memcmp(readdir->rd_verf.data, zeroverf.data, NFS4_VERIFIER_SIZE)))
@@ -754,8 +756,9 @@ _nfsd4_verify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (status)
 		return status;
 
-	if ((verify->ve_bmval[0] & ~NFSD_SUPPORTED_ATTRS_WORD0)
-	    || (verify->ve_bmval[1] & ~NFSD_SUPPORTED_ATTRS_WORD1))
+	if ((verify->ve_bmval[0] & ~nfsd_suppattrs0(cstate->minorversion))
+	    || (verify->ve_bmval[1] & ~nfsd_suppattrs1(cstate->minorversion))
+	    || (verify->ve_bmval[2] & ~nfsd_suppattrs2(cstate->minorversion)))
 		return nfserr_attrnotsupp;
 	if ((verify->ve_bmval[0] & FATTR4_WORD0_RDATTR_ERROR)
 	    || (verify->ve_bmval[1] & NFSD_WRITEONLY_ATTRS_WORD1))
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 96f3eeb..079626c 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -236,6 +236,7 @@ nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 
 	bmval[0] = 0;
 	bmval[1] = 0;
+	bmval[2] = 0;
 
 	READ_BUF(4);
 	READ32(bmlen);
@@ -247,13 +248,16 @@ nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 		READ32(bmval[0]);
 	if (bmlen > 1)
 		READ32(bmval[1]);
+	if (bmlen > 2)
+		READ32(bmval[2]);
 
 	DECODE_TAIL;
 }
 
 static u32 nfsd_attrmask[] = {
 	NFSD_WRITEABLE_ATTRS_WORD0,
-	NFSD_WRITEABLE_ATTRS_WORD1
+	NFSD_WRITEABLE_ATTRS_WORD1,
+	NFSD_WRITEABLE_ATTRS_WORD2
 };
 
 static __be32
@@ -274,9 +278,12 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
 	 * According to spec, unsupported attributes return ERR_ATTRNOTSUPP;
 	 * read-only attributes return ERR_INVAL.
 	 */
-	if ((bmval[0] & ~NFSD_SUPPORTED_ATTRS_WORD0) || (bmval[1] & ~NFSD_SUPPORTED_ATTRS_WORD1))
+	if ((bmval[0] & ~nfsd_suppattrs0(argp->minorversion)) ||
+	    (bmval[1] & ~nfsd_suppattrs1(argp->minorversion)) ||
+	    (bmval[2] & ~nfsd_suppattrs2(argp->minorversion)))
 		return nfserr_attrnotsupp;
-	if ((bmval[0] & ~writable[0]) || (bmval[1] & ~writable[1]))
+	if ((bmval[0] & ~writable[0]) || (bmval[1] & ~writable[1]) ||
+	    (bmval[2] & ~writable[2]))
 		return nfserr_inval;
 
 	READ_BUF(4);
@@ -411,6 +418,7 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
 			goto xdr_error;
 		}
 	}
+	BUG_ON(bmval[2]);	/* no such writeable attr supported yet */
 	if (len != expected_len)
 		goto xdr_error;
 
@@ -1732,6 +1740,7 @@ nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
 {
 	u32 bmval0 = bmval[0];
 	u32 bmval1 = bmval[1];
+	u32 bmval2 = bmval[2];
 	struct kstat stat;
 	struct svc_fh tempfh;
 	struct kstatfs statfs;
@@ -1745,12 +1754,16 @@ nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
 	int err;
 	int aclsupport = 0;
 	struct nfs4_acl *acl = NULL;
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
+	u32 minorversion = resp->cstate.minorversion;
 
 	BUG_ON(bmval1 & NFSD_WRITEONLY_ATTRS_WORD1);
-	BUG_ON(bmval0 & ~NFSD_SUPPORTED_ATTRS_WORD0);
-	BUG_ON(bmval1 & ~NFSD_SUPPORTED_ATTRS_WORD1);
+	BUG_ON(bmval0 & ~nfsd_suppattrs0(minorversion));
+	BUG_ON(bmval1 & ~nfsd_suppattrs1(minorversion));
+	BUG_ON(bmval2 & ~nfsd_suppattrs2(minorversion));
 
 	if (exp->ex_fslocs.migrated) {
+		BUG_ON(bmval[2]);
 		status = fattr_handle_absent_fs(&bmval0, &bmval1, &rdattr_err);
 		if (status)
 			goto out;
@@ -1796,22 +1809,42 @@ nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
 	if ((buflen -= 16) < 0)
 		goto out_resource;
 
-	WRITE32(2);
-	WRITE32(bmval0);
-	WRITE32(bmval1);
+	if (unlikely(bmval2)) {
+		WRITE32(3);
+		WRITE32(bmval0);
+		WRITE32(bmval1);
+		WRITE32(bmval2);
+	} else if (likely(bmval1)) {
+		WRITE32(2);
+		WRITE32(bmval0);
+		WRITE32(bmval1);
+	} else {
+		WRITE32(1);
+		WRITE32(bmval0);
+	}
 	attrlenp = p++;                /* to be backfilled later */
 
 	if (bmval0 & FATTR4_WORD0_SUPPORTED_ATTRS) {
-		u32 word0 = NFSD_SUPPORTED_ATTRS_WORD0;
+		u32 word0 = nfsd_suppattrs0(minorversion);
+		u32 word1 = nfsd_suppattrs1(minorversion);
+		u32 word2 = nfsd_suppattrs2(minorversion);
+
 		if ((buflen -= 12) < 0)
 			goto out_resource;
 		if (!aclsupport)
 			word0 &= ~FATTR4_WORD0_ACL;
 		if (!exp->ex_fslocs.locations)
 			word0 &= ~FATTR4_WORD0_FS_LOCATIONS;
-		WRITE32(2);
-		WRITE32(word0);
-		WRITE32(NFSD_SUPPORTED_ATTRS_WORD1);
+		if (!word2) {
+			WRITE32(2);
+			WRITE32(word0);
+			WRITE32(word1);
+		} else {
+			WRITE32(3);
+			WRITE32(word0);
+			WRITE32(word1);
+			WRITE32(word2);
+		}
 	}
 	if (bmval0 & FATTR4_WORD0_TYPE) {
 		if ((buflen -= 4) < 0)
@@ -2121,6 +2154,8 @@ out_acl:
 		}
 		WRITE64(stat.ino);
 	}
+	BUG_ON(bmval2);	/* FIXME: not implemented yet */
+
 	*attrlenp = htonl((char *)p - (char *)attrlenp - 4);
 	*countp = p - buffer;
 	status = nfs_ok;
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index d0fe9b4..3dccb76 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -345,7 +345,7 @@ extern struct timeval	nfssvc_boot;
  *    TIME_BACKUP   (unlikely to be supported any time soon)
  *    TIME_CREATE   (unlikely to be supported any time soon)
  */
-#define NFSD_SUPPORTED_ATTRS_WORD0                                                          \
+#define NFSD4_SUPPORTED_ATTRS_WORD0                                                         \
 (FATTR4_WORD0_SUPPORTED_ATTRS   | FATTR4_WORD0_TYPE         | FATTR4_WORD0_FH_EXPIRE_TYPE   \
  | FATTR4_WORD0_CHANGE          | FATTR4_WORD0_SIZE         | FATTR4_WORD0_LINK_SUPPORT     \
  | FATTR4_WORD0_SYMLINK_SUPPORT | FATTR4_WORD0_NAMED_ATTR   | FATTR4_WORD0_FSID             \
@@ -357,7 +357,7 @@ extern struct timeval	nfssvc_boot;
  | FATTR4_WORD0_MAXFILESIZE     | FATTR4_WORD0_MAXLINK      | FATTR4_WORD0_MAXNAME          \
  | FATTR4_WORD0_MAXREAD         | FATTR4_WORD0_MAXWRITE     | FATTR4_WORD0_ACL)
 
-#define NFSD_SUPPORTED_ATTRS_WORD1                                                          \
+#define NFSD4_SUPPORTED_ATTRS_WORD1                                                         \
 (FATTR4_WORD1_MODE              | FATTR4_WORD1_NO_TRUNC     | FATTR4_WORD1_NUMLINKS         \
  | FATTR4_WORD1_OWNER	        | FATTR4_WORD1_OWNER_GROUP  | FATTR4_WORD1_RAWDEV           \
  | FATTR4_WORD1_SPACE_AVAIL     | FATTR4_WORD1_SPACE_FREE   | FATTR4_WORD1_SPACE_TOTAL      \
@@ -365,11 +365,54 @@ extern struct timeval	nfssvc_boot;
  | FATTR4_WORD1_TIME_DELTA   | FATTR4_WORD1_TIME_METADATA    \
  | FATTR4_WORD1_TIME_MODIFY     | FATTR4_WORD1_TIME_MODIFY_SET | FATTR4_WORD1_MOUNTED_ON_FILEID)
 
+#define NFSD4_SUPPORTED_ATTRS_WORD2 0
+
+#define NFSD4_1_SUPPORTED_ATTRS_WORD0 \
+	NFSD4_SUPPORTED_ATTRS_WORD0
+
+#define NFSD4_1_SUPPORTED_ATTRS_WORD1 \
+	NFSD4_SUPPORTED_ATTRS_WORD1
+
+#define NFSD4_1_SUPPORTED_ATTRS_WORD2 \
+	NFSD4_SUPPORTED_ATTRS_WORD2
+
 #if defined(CONFIG_NFSD_V4_1)
 
 /* The percent of nr_free_buffer_pages used by the V4.1 server DRC */
 #define NFSD_DRC_SIZE_SHIFT	7
 
+static inline u32 nfsd_suppattrs0(u32 minorversion)
+{
+	return minorversion ? NFSD4_1_SUPPORTED_ATTRS_WORD0
+			    : NFSD4_SUPPORTED_ATTRS_WORD0;
+}
+
+static inline u32 nfsd_suppattrs1(u32 minorversion)
+{
+	return minorversion ? NFSD4_1_SUPPORTED_ATTRS_WORD1
+			    : NFSD4_SUPPORTED_ATTRS_WORD1;
+}
+
+static inline u32 nfsd_suppattrs2(u32 minorversion)
+{
+	return minorversion ? NFSD4_1_SUPPORTED_ATTRS_WORD2
+			    : NFSD4_SUPPORTED_ATTRS_WORD2;
+}
+#else  /* CONFIG_NFSD_V4_1 */
+static inline u32 nfsd_suppattrs0(u32 minorversion)
+{
+	return NFSD4_SUPPORTED_ATTRS_WORD0;
+}
+
+static inline u32 nfsd_suppattrs1(u32 minorversion)
+{
+	return NFSD4_SUPPORTED_ATTRS_WORD1;
+}
+
+static inline u32 nfsd_suppattrs2(u32 minorversion)
+{
+	return NFSD4_SUPPORTED_ATTRS_WORD2;
+}
 #endif /* CONFIG_NFSD_V4_1 */
 
 /* These will return ERR_INVAL if specified in GETATTR or READDIR. */
@@ -382,6 +425,7 @@ extern struct timeval	nfssvc_boot;
 #define NFSD_WRITEABLE_ATTRS_WORD1                                                          \
 (FATTR4_WORD1_MODE              | FATTR4_WORD1_OWNER         | FATTR4_WORD1_OWNER_GROUP     \
  | FATTR4_WORD1_TIME_ACCESS_SET | FATTR4_WORD1_TIME_MODIFY_SET)
+#define NFSD_WRITEABLE_ATTRS_WORD2 0
 
 #endif /* CONFIG_NFSD_V4 */
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 881ff67..d7a8050 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -101,7 +101,7 @@ struct nfsd4_create {
 			u32 specdata2;
 		} dev;    /* NF4BLK, NF4CHR */
 	} u;
-	u32		cr_bmval[2];        /* request */
+	u32		cr_bmval[3];        /* request */
 	struct iattr	cr_iattr;           /* request */
 	struct nfsd4_change_info  cr_cinfo; /* response */
 	struct nfs4_acl *cr_acl;
@@ -116,7 +116,7 @@ struct nfsd4_delegreturn {
 };
 
 struct nfsd4_getattr {
-	u32		ga_bmval[2];        /* request */
+	u32		ga_bmval[3];        /* request */
 	struct svc_fh	*ga_fhp;            /* response */
 };
 
@@ -217,7 +217,7 @@ struct nfsd4_open {
 	stateid_t       op_delegate_stateid; /* request - response */
 	u32		op_create;     	    /* request */
 	u32		op_createmode;      /* request */
-	u32		op_bmval[2];        /* request */
+	u32		op_bmval[3];        /* request */
 	union {                             /* request */
 		struct iattr	iattr;                      /* UNCHECKED4,GUARDED4 */
 		nfs4_verifier	verf;                                /* EXCLUSIVE4 */
@@ -270,7 +270,7 @@ struct nfsd4_readdir {
 	nfs4_verifier	rd_verf;            /* request */
 	u32		rd_dircount;        /* request */
 	u32		rd_maxcount;        /* request */
-	u32		rd_bmval[2];        /* request */
+	u32		rd_bmval[3];        /* request */
 	struct svc_rqst *rd_rqstp;          /* response */
 	struct svc_fh * rd_fhp;             /* response */
 
@@ -312,7 +312,7 @@ struct nfsd4_secinfo {
 
 struct nfsd4_setattr {
 	stateid_t	sa_stateid;         /* request */
-	u32		sa_bmval[2];        /* request */
+	u32		sa_bmval[3];        /* request */
 	struct iattr	sa_iattr;           /* request */
 	struct nfs4_acl *sa_acl;
 };
@@ -338,7 +338,7 @@ struct nfsd4_setclientid_confirm {
 
 /* also used for NVERIFY */
 struct nfsd4_verify {
-	u32		ve_bmval[2];        /* request */
+	u32		ve_bmval[3];        /* request */
 	u32		ve_attrlen;         /* request */
 	char *		ve_attrval;         /* request */
 };
-- 
1.6.2.1

[PATCH 45/47] nfsd41: SUPPATTR_EXCLCREAT attribute

[Benny Halevy]

Return bitmask for supported EXCLUSIVE4_1 create attributes.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c         |    7 ++++++-
 include/linux/nfs4.h      |    2 ++
 include/linux/nfsd/nfsd.h |   14 +++++++++++++-
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 079626c..288332c 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -2154,7 +2154,12 @@ out_acl:
 		}
 		WRITE64(stat.ino);
 	}
-	BUG_ON(bmval2);	/* FIXME: not implemented yet */
+	if (bmval2 & FATTR4_WORD2_SUPPATTR_EXCLCREAT) {
+		WRITE32(3);
+		WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD0);
+		WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD1);
+		WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD2);
+	}
 
 	*attrlenp = htonl((char *)p - (char *)attrlenp - 4);
 	*countp = p - buffer;
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index a34683e..735c7d8 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -404,6 +404,8 @@ enum lock_type4 {
 #define FATTR4_WORD0_UNIQUE_HANDLES     (1UL << 9)
 #define FATTR4_WORD0_LEASE_TIME         (1UL << 10)
 #define FATTR4_WORD0_RDATTR_ERROR       (1UL << 11)
+/* Mandatory in NFSv4.1 */
+#define FATTR4_WORD2_SUPPATTR_EXCLCREAT (1UL << 11)
 
 /* Recommended Attributes */
 #define FATTR4_WORD0_ACL                (1UL << 12)
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index 3dccb76..b9e6682 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -374,7 +374,7 @@ extern struct timeval	nfssvc_boot;
 	NFSD4_SUPPORTED_ATTRS_WORD1
 
 #define NFSD4_1_SUPPORTED_ATTRS_WORD2 \
-	NFSD4_SUPPORTED_ATTRS_WORD2
+	(NFSD4_SUPPORTED_ATTRS_WORD2 | FATTR4_WORD2_SUPPATTR_EXCLCREAT)
 
 #if defined(CONFIG_NFSD_V4_1)
 
@@ -427,6 +427,18 @@ static inline u32 nfsd_suppattrs2(u32 minorversion)
  | FATTR4_WORD1_TIME_ACCESS_SET | FATTR4_WORD1_TIME_MODIFY_SET)
 #define NFSD_WRITEABLE_ATTRS_WORD2 0
 
+#define NFSD_SUPPATTR_EXCLCREAT_WORD0 \
+	NFSD_WRITEABLE_ATTRS_WORD0
+/*
+ * we currently store the exclusive create verifier in the v_{a,m}time
+ * attributes so the client can't set these at create time using EXCLUSIVE4_1
+ */
+#define NFSD_SUPPATTR_EXCLCREAT_WORD1 \
+	(NFSD_WRITEABLE_ATTRS_WORD1 & \
+	 ~(FATTR4_WORD1_TIME_ACCESS_SET | FATTR4_WORD1_TIME_MODIFY_SET))
+#define NFSD_SUPPATTR_EXCLCREAT_WORD2 \
+	NFSD_WRITEABLE_ATTRS_WORD2
+
 #endif /* CONFIG_NFSD_V4 */
 
 #endif /* LINUX_NFSD_NFSD_H */
-- 
1.6.2.1

[PATCH 46/47] nfsd41: CREATE_EXCLUSIVE4_1

[Benny Halevy]

Implement the CREATE_EXCLUSIVE4_1 open mode conforming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

This mode allows the client to atomically create a file
if it doesn't exist while setting some of its attributes.

It must be implemented if the server supports persistent
reply cache and/or pnfs.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   15 +++++++++++++++
 fs/nfsd/nfs4xdr.c         |   17 +++++++++++++++++
 include/linux/nfs4.h      |    8 +++++++-
 include/linux/nfsd/xdr4.h |   10 ++++------
 4 files changed, 43 insertions(+), 7 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f3b560f..6d7d7c5 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -93,6 +93,21 @@ do_open_lookup(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_o
 	open->op_truncate = 0;
 
 	if (open->op_create) {
+		/* FIXME: check session persistence and pnfs flags.
+		 * The nfsv4.1 spec requires the following semantics:
+		 *
+		 * Persistent   | pNFS   | Server REQUIRED | Client Allowed
+		 * Reply Cache  | server |                 |
+		 * -------------+--------+-----------------+--------------------
+		 * no           | no     | EXCLUSIVE4_1    | EXCLUSIVE4_1
+		 *              |        |                 | (SHOULD)
+		 *              |        | and EXCLUSIVE4  | or EXCLUSIVE4
+		 *              |        |                 | (SHOULD NOT)
+		 * no           | yes    | EXCLUSIVE4_1    | EXCLUSIVE4_1
+		 * yes          | no     | GUARDED4        | GUARDED4
+		 * yes          | yes    | GUARDED4        | GUARDED4
+		 */
+
 		/*
 		 * Note: create modes (UNCHECKED,GUARDED...) are the same
 		 * in NFSv4 as in v3.
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 288332c..0084459 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -260,6 +260,12 @@ static u32 nfsd_attrmask[] = {
 	NFSD_WRITEABLE_ATTRS_WORD2
 };
 
+static u32 nfsd41_ex_attrmask[] = {
+	NFSD_SUPPATTR_EXCLCREAT_WORD0,
+	NFSD_SUPPATTR_EXCLCREAT_WORD1,
+	NFSD_SUPPATTR_EXCLCREAT_WORD2
+};
+
 static __be32
 nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
 		   struct iattr *iattr, struct nfs4_acl **acl)
@@ -684,6 +690,17 @@ nfsd4_decode_open(struct nfsd4_compoundargs *argp, struct nfsd4_open *open)
 			READ_BUF(8);
 			COPYMEM(open->op_verf.data, 8);
 			break;
+		case NFS4_CREATE_EXCLUSIVE4_1:
+			if (argp->minorversion < 1)
+				goto xdr_error;
+			READ_BUF(8);
+			COPYMEM(open->op_verf.data, 8);
+			status = nfsd4_decode_fattr(argp, open->op_bmval,
+				nfsd41_ex_attrmask, &open->op_iattr,
+				&open->op_acl);
+			if (status)
+				goto out;
+			break;
 		default:
 			goto xdr_error;
 		}
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 735c7d8..5756359 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -368,7 +368,13 @@ enum opentype4 {
 enum createmode4 {
 	NFS4_CREATE_UNCHECKED = 0,
 	NFS4_CREATE_GUARDED = 1,
-	NFS4_CREATE_EXCLUSIVE = 2
+	NFS4_CREATE_EXCLUSIVE = 2,
+	/*
+	 * New to NFSv4.1. If session is persistent,
+	 * GUARDED4 MUST be used. Otherwise, use
+	 * EXCLUSIVE4_1 instead of EXCLUSIVE4.
+	 */
+	NFS4_CREATE_EXCLUSIVE4_1 = 3
 };
 
 enum limit_by4 {
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index d7a8050..7ed9955 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -218,10 +218,8 @@ struct nfsd4_open {
 	u32		op_create;     	    /* request */
 	u32		op_createmode;      /* request */
 	u32		op_bmval[3];        /* request */
-	union {                             /* request */
-		struct iattr	iattr;                      /* UNCHECKED4,GUARDED4 */
-		nfs4_verifier	verf;                                /* EXCLUSIVE4 */
-	} u;
+	struct iattr	iattr;              /* UNCHECKED4, GUARDED4, EXCLUSIVE4_1 */
+	nfs4_verifier	verf;               /* EXCLUSIVE4 */
 	clientid_t	op_clientid;        /* request */
 	struct xdr_netobj op_owner;           /* request */
 	u32		op_seqid;           /* request */
@@ -235,8 +233,8 @@ struct nfsd4_open {
 	struct nfs4_stateowner *op_stateowner; /* used during processing */
 	struct nfs4_acl *op_acl;
 };
-#define op_iattr	u.iattr
-#define op_verf		u.verf
+#define op_iattr	iattr
+#define op_verf		verf
 
 struct nfsd4_open_confirm {
 	stateid_t	oc_req_stateid		/* request */;
-- 
1.6.2.1

[PATCH 47/47] nfsd41: Documentation/filesystems/nfs41-server.txt

[Benny Halevy]

Initial nfs41 server write up describing the status of the linux
server implementation.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 Documentation/filesystems/nfs41-server.txt |  148 ++++++++++++++++++++++++++++
 1 files changed, 148 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/filesystems/nfs41-server.txt

diff --git a/Documentation/filesystems/nfs41-server.txt b/Documentation/filesystems/nfs41-server.txt
new file mode 100644
index 0000000..e62cd13
--- /dev/null
+++ b/Documentation/filesystems/nfs41-server.txt
@@ -0,0 +1,148 @@
+NFSv4.1 Server Implementation
+
+The NFSv4 minorversion 1 (NFSv4.1) implementation in nfsd is based
+on the latest NFSv4.1 Internet Draft:
+http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29
+
+If you're a developer or just eager to test the code, set
+CONFIG_NFSD_V4_1 to 'y' in your kernel configuration file.
+This will allow the server to serve both NFSv4 and NFSv4.1
+clients concurrently.
+
+From the many new features in NFSv4.1 the current implementation
+focuses on the mandatory-to-implement NFSv4.1 Sessions, providing
+"exactly once" semantics and better control and throttling of the
+resources allocated for each client.
+
+Other NFSv4.1 features, Parallel NFS operations in particular,
+are still under development out of tree.
+See http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design
+for more information.
+
+The table below, taken from the NFSv4.1 document, lists
+the operations that are mandatory to implement (REQ), optional
+(OPT), and NFSv4.0 operations that are required not to implement (MNI)
+in minor version 1.  The first column indicates the operations that
+are not supported yet by the linux server implementation.
+
+The OPTIONAL features identified and their abbreviations are as follows:
+	pNFS	Parallel NFS
+	FDELG	File Delegations
+	DDELG	Directory Delegations
+
+The following abbreviations indicate the linux server implementation status.
+	I	Implemented NFSv4.1 operations.
+	NS	Not Supported.
+	NS*	unimplemented optional feature.
+	P	pNFS features implemented out of tree.
+	PNS	pNFS features that are not supported yet (out of tree).
+
+Operations
+
+   +----------------------+------------+--------------+----------------+
+   | Operation            | REQ, REC,  | Feature      | Definition     |
+   |                      | OPT, or    | (REQ, REC,   |                |
+   |                      | MNI        | or OPT)      |                |
+   +----------------------+------------+--------------+----------------+
+   | ACCESS               | REQ        |              | Section 18.1   |
+NS | BACKCHANNEL_CTL      | REQ        |              | Section 18.33  |
+NS | BIND_CONN_TO_SESSION | REQ        |              | Section 18.34  |
+   | CLOSE                | REQ        |              | Section 18.2   |
+   | COMMIT               | REQ        |              | Section 18.3   |
+   | CREATE               | REQ        |              | Section 18.4   |
+I  | CREATE_SESSION       | REQ        |              | Section 18.36  |
+NS*| DELEGPURGE           | OPT        | FDELG (REQ)  | Section 18.5   |
+   | DELEGRETURN          | OPT        | FDELG,       | Section 18.6   |
+   |                      |            | DDELG, pNFS  |                |
+   |                      |            | (REQ)        |                |
+NS | DESTROY_CLIENTID     | REQ        |              | Section 18.50  |
+I  | DESTROY_SESSION      | REQ        |              | Section 18.37  |
+I  | EXCHANGE_ID          | REQ        |              | Section 18.35  |
+NS | FREE_STATEID         | REQ        |              | Section 18.38  |
+   | GETATTR              | REQ        |              | Section 18.7   |
+P  | GETDEVICEINFO        | OPT        | pNFS (REQ)   | Section 18.40  |
+P  | GETDEVICELIST        | OPT        | pNFS (OPT)   | Section 18.41  |
+   | GETFH                | REQ        |              | Section 18.8   |
+NS*| GET_DIR_DELEGATION   | OPT        | DDELG (REQ)  | Section 18.39  |
+P  | LAYOUTCOMMIT         | OPT        | pNFS (REQ)   | Section 18.42  |
+P  | LAYOUTGET            | OPT        | pNFS (REQ)   | Section 18.43  |
+P  | LAYOUTRETURN         | OPT        | pNFS (REQ)   | Section 18.44  |
+   | LINK                 | OPT        |              | Section 18.9   |
+   | LOCK                 | REQ        |              | Section 18.10  |
+   | LOCKT                | REQ        |              | Section 18.11  |
+   | LOCKU                | REQ        |              | Section 18.12  |
+   | LOOKUP               | REQ        |              | Section 18.13  |
+   | LOOKUPP              | REQ        |              | Section 18.14  |
+   | NVERIFY              | REQ        |              | Section 18.15  |
+   | OPEN                 | REQ        |              | Section 18.16  |
+NS*| OPENATTR             | OPT        |              | Section 18.17  |
+   | OPEN_CONFIRM         | MNI        |              | N/A            |
+   | OPEN_DOWNGRADE       | REQ        |              | Section 18.18  |
+   | PUTFH                | REQ        |              | Section 18.19  |
+   | PUTPUBFH             | REQ        |              | Section 18.20  |
+   | PUTROOTFH            | REQ        |              | Section 18.21  |
+   | READ                 | REQ        |              | Section 18.22  |
+   | READDIR              | REQ        |              | Section 18.23  |
+   | READLINK             | OPT        |              | Section 18.24  |
+NS | RECLAIM_COMPLETE     | REQ        |              | Section 18.51  |
+   | RELEASE_LOCKOWNER    | MNI        |              | N/A            |
+   | REMOVE               | REQ        |              | Section 18.25  |
+   | RENAME               | REQ        |              | Section 18.26  |
+   | RENEW                | MNI        |              | N/A            |
+   | RESTOREFH            | REQ        |              | Section 18.27  |
+   | SAVEFH               | REQ        |              | Section 18.28  |
+   | SECINFO              | REQ        |              | Section 18.29  |
+NS | SECINFO_NO_NAME      | REC        | pNFS files   | Section 18.45, |
+   |                      |            | layout (REQ) | Section 13.12  |
+I  | SEQUENCE             | REQ        |              | Section 18.46  |
+   | SETATTR              | REQ        |              | Section 18.30  |
+   | SETCLIENTID          | MNI        |              | N/A            |
+   | SETCLIENTID_CONFIRM  | MNI        |              | N/A            |
+NS | SET_SSV              | REQ        |              | Section 18.47  |
+NS | TEST_STATEID         | REQ        |              | Section 18.48  |
+   | VERIFY               | REQ        |              | Section 18.31  |
+NS*| WANT_DELEGATION      | OPT        | FDELG (OPT)  | Section 18.49  |
+   | WRITE                | REQ        |              | Section 18.32  |
+
+Callback Operations
+
+   +-------------------------+-----------+-------------+---------------+
+   | Operation               | REQ, REC, | Feature     | Definition    |
+   |                         | OPT, or   | (REQ, REC,  |               |
+   |                         | MNI       | or OPT)     |               |
+   +-------------------------+-----------+-------------+---------------+
+   | CB_GETATTR              | OPT       | FDELG (REQ) | Section 20.1  |
+P  | CB_LAYOUTRECALL         | OPT       | pNFS (REQ)  | Section 20.3  |
+NS*| CB_NOTIFY               | OPT       | DDELG (REQ) | Section 20.4  |
+P  | CB_NOTIFY_DEVICEID      | OPT       | pNFS (OPT)  | Section 20.12 |
+NS*| CB_NOTIFY_LOCK          | OPT       |             | Section 20.11 |
+NS*| CB_PUSH_DELEG           | OPT       | FDELG (OPT) | Section 20.5  |
+   | CB_RECALL               | OPT       | FDELG,      | Section 20.2  |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+NS*| CB_RECALL_ANY           | OPT       | FDELG,      | Section 20.6  |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+NS | CB_RECALL_SLOT          | REQ       |             | Section 20.8  |
+NS*| CB_RECALLABLE_OBJ_AVAIL | OPT       | DDELG, pNFS | Section 20.7  |
+   |                         |           | (REQ)       |               |
+I  | CB_SEQUENCE             | OPT       | FDELG,      | Section 20.9  |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+NS*| CB_WANTS_CANCELLED      | OPT       | FDELG,      | Section 20.10 |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+   +-------------------------+-----------+-------------+---------------+
+
+Implementation notes:
+
+EXCHANGE_ID:
+* only SP4_NONE state protection supported
+* implementation ids are ignored
+
+CREATE_SESSION:
+* backchannel attributes are ignored
+* backchannel security parameters are ignored
+
+SEQUENCE:
+* no support for dynamic slot table renegotiation (optional)
-- 
1.6.2.1

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> Hi Bruce,
> 
> Here's the latest server patches implementing the NFSv4.1
> Sessions features.
> 
> This patchset is based over your for-2.6.30 branch
> and is also available from
> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30

There's a few merge conflicts with my current for-2.6.30--probably my
fault for not pushing that out recently enough, apologies.  Would you
mind updating?

--b.

> (tagged as nfsd41-for-2.6.30-2009-03-27)
> 
> The implementation status highlights are captured in
> Documentation/filesystems/nfs41-server.txt.
> The gist of it is that the implementation is functional
> to the level of passing Connectathon tests with v2,v3,v4,v4.1
> (tested with CONFIG_NFS_V4_1 defined to 'y')
> 
> The patches are listed below.
> 
> Thanks for your review and help with this project so far.
> Kudos to all the contributors (Authors/SOBs: Andy Adamson,
> Dean Hildebrand, Marc Eshel, Mike Sager, Ricardo Labiaga, and
> myself)  This is a pretty significant milestone for us
> (well, the real one will be when all this ends up working
> flawlessly in the kernel. :)
> 
> Benny
> 
> [PATCH 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY
> [PATCH 02/47] sunrpc: add cl_private field to struct rpc_clnt
> [PATCH 03/47] nfsd: embed nfsd4_current_state in nfsd4_compoundres
> [PATCH 04/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state
> [PATCH 05/47] nfs41: common protocol definitions
> [PATCH 06/47] nfsd41: change NFSERR_REPLAY_ME
> [PATCH 07/47] nfsd41: Add Kconfig symbols for NFSv4.1
> [PATCH 08/47] nfsd41: define nfs41 error codes
> [PATCH 09/47] nfsd41: sessions basic data types
> [PATCH 10/47] nfsd41: introduce nfs4_client cl_sessions list
> [PATCH 11/47] nfsd41: release_session when client is expired
> [PATCH 12/47] nfsd41: sessionid hashing
> [PATCH 13/47] nfsd41: xdr infrastructure
> [PATCH 14/47] nfsd: remove nfsd4_ops array size
> [PATCH 15/47] nfsd41: proc stubs
> [PATCH 16/47] nfsd41: exchange_id operation
> [PATCH 17/47] nfsd41: match clientid establishment method
> [PATCH 18/47] nfsd41: sequence operation
> [PATCH 19/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules
> [PATCH 20/47] nfsd41: DRC save, restore, and clear functions
> [PATCH 21/47] nfsd41: hard page limit for DRC
> [PATCH 22/47] nfsd41: nfsd DRC logic
> [PATCH 23/47] nfsd41: clear DRC cache on free_session
> [PATCH 24/47] nfsd41: create_session operation
> [PATCH 25/47] nfsd41: Add a create session replay cache
> [PATCH 26/47] nfsd41: non-page DRC for solo sequence responses
> [PATCH 27/47] nfsd41: destroy_session operation
> [PATCH 28/47] nfsd41: stateid handling
> [PATCH 29/47] nfsd41: check encode size for sessions maxresponse cached
> [PATCH 30/47] nfsd41: clientid handling
> [PATCH 31/47] nfsd41: access_valid
> [PATCH 32/47] nfsd41: add OPEN4_SHARE_ACCESS_WANT nfs4_stateid bmap
> [PATCH 33/47] nfsd41: provide support for minor version 1 at rpc level
> [PATCH 34/47] nfsd: cleanup nfs4.0 callback encode routines
> [PATCH 35/47] nfsd: minorversion support for the back channel
> [PATCH 36/47] nfsd41: sunrpc: Added rpc server-side backchannel handling
> [PATCH 37/47] nfsd41: callback infrastructure
> [PATCH 38/47] nfsd41: Remember the auth flavor to use for callbacks
> [PATCH 39/47] nfsd41: introduce cl_cb_mutex
> [PATCH 40/47] nfsd41: cb_sequence callback
> [PATCH 41/47] nfsd41: introduce nfs4_cb_call_sync for nfs4 and nfs41
> [PATCH 42/47] nfsd41: cb_recall callback
> [PATCH 43/47] nfsd41: pass writable attrs mask to nfsd4_decode_fattr
> [PATCH 44/47] nfsd41: support for 3-word long attribute bitmask
> [PATCH 45/47] nfsd41: SUPPATTR_EXCLCREAT attribute
> [PATCH 46/47] nfsd41: CREATE_EXCLUSIVE4_1
> [PATCH 47/47] nfsd41: Documentation/filesystems/nfs41-server.txt
> 

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Benny Halevy]

On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
>> Hi Bruce,
>>
>> Here's the latest server patches implementing the NFSv4.1
>> Sessions features.
>>
>> This patchset is based over your for-2.6.30 branch
>> and is also available from
>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> 
> There's a few merge conflicts with my current for-2.6.30--probably my
> fault for not pushing that out recently enough, apologies.  Would you
> mind updating?
> 
> --b.
> 

Sure. Here's a rebased version with two minor changes (see below)

Benny

[PATCH v2 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY
commit message edited.

[PATCH v2 02/47] nfsd: embed nfsd4_current_state in nfsd4_compoundres
[PATCH v2 03/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state
[PATCH v2 04/47] nfs41: common protocol definitions
[PATCH v2 05/47] nfsd41: change NFSERR_REPLAY_ME
[PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1
[PATCH v2 07/47] nfsd41: define nfs41 error codes
[PATCH v2 08/47] nfsd41: sessions basic data types
[PATCH v2 09/47] nfsd41: introduce nfs4_client cl_sessions list
[PATCH v2 10/47] nfsd41: release_session when client is expired
[PATCH v2 11/47] nfsd41: sessionid hashing
[PATCH v2 12/47] nfsd41: xdr infrastructure
[PATCH v2 13/47] nfsd: remove nfsd4_ops array size
[PATCH v2 14/47] nfsd41: proc stubs
[PATCH v2 15/47] nfsd41: exchange_id operation
[PATCH v2 16/47] nfsd41: match clientid establishment method
[PATCH v2 17/47] nfsd41: sequence operation
[PATCH v2 18/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules
[PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions
[PATCH v2 20/47] nfsd41: hard page limit for DRC
[PATCH v2 21/47] nfsd41: nfsd DRC logic
[PATCH v2 22/47] nfsd41: clear DRC cache on free_session
[PATCH v2 23/47] nfsd41: create_session operation
[PATCH v2 24/47] nfsd41: Add a create session replay cache
[PATCH v2 25/47] nfsd41: non-page DRC for solo sequence responses
[PATCH v2 26/47] nfsd41: destroy_session operation
[PATCH v2 27/47] nfsd41: stateid handling
[PATCH v2 28/47] nfsd41: check encode size for sessions maxresponse cached
[PATCH v2 29/47] nfsd41: clientid handling
[PATCH v2 30/47] nfsd41: access_valid
[PATCH v2 31/47] nfsd41: add OPEN4_SHARE_ACCESS_WANT nfs4_stateid bmap
[PATCH v2 32/47] nfsd41: provide support for minor version 1 at rpc level
[PATCH v2 33/47] nfsd: cleanup nfs4.0 callback encode routines
[PATCH v2 34/47] sunrpc: add cl_private field to struct rpc_clnt
patch moved forward, right before first use.

[PATCH v2 35/47] nfsd: minorversion support for the back channel
[PATCH v2 36/47] nfsd41: sunrpc: Added rpc server-side backchannel handling
[PATCH v2 37/47] nfsd41: callback infrastructure
[PATCH v2 38/47] nfsd41: Remember the auth flavor to use for callbacks
[PATCH v2 39/47] nfsd41: introduce cl_cb_mutex
[PATCH v2 40/47] nfsd41: cb_sequence callback
[PATCH v2 41/47] nfsd41: introduce nfs4_cb_call_sync for nfs4 and nfs41
[PATCH v2 42/47] nfsd41: cb_recall callback
[PATCH v2 43/47] nfsd41: pass writable attrs mask to nfsd4_decode_fattr
[PATCH v2 44/47] nfsd41: support for 3-word long attribute bitmask
[PATCH v2 45/47] nfsd41: SUPPATTR_EXCLCREAT attribute
[PATCH v2 46/47] nfsd41: CREATE_EXCLUSIVE4_1
[PATCH v2 47/47] nfsd41: Documentation/filesystems/nfs41-server.txt

[PATCH v2 01/47] nfsd: don't use the deferral service, return NFS4ERR_DELAY

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

On an NFSv4.1 server cache miss that causes an upcall, NFS4ERR_DELAY will be
returned. It is up to the NFSv4.1 client to resend only the operations that
have not been processed.

Initialize rq_usedeferral to 1 in svc_process(). It sill be turned off in
nfsd4_proc_compound() only when NFSv4.1 Sessions are used.

Note: this isn't an adequate solution on its own. It's acceptable as a way
to get some minimal 4.1 up and working, but we're going to have to find a
way to avoid returning DELAY in all common cases before 4.1 can really be
considered ready.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: reverse rq_nodeferral negative logic]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[sunrpc: initialize rq_usedeferral]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c         |    8 ++++++++
 include/linux/sunrpc/svc.h |    1 +
 net/sunrpc/svc.c           |    2 ++
 net/sunrpc/svc_xprt.c      |    2 +-
 4 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f156b85..7839654 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -873,6 +873,8 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	resp->tag = args->tag;
 	resp->opcnt = 0;
 	resp->rqstp = rqstp;
+	/* Use the deferral mechanism only for NFSv4.0 compounds */
+	rqstp->rq_usedeferral = (args->minorversion == 0);
 
 	/*
 	 * According to RFC3010, this takes precedence over all other errors.
@@ -957,10 +959,16 @@ encode_op:
 
 		nfsd4_increment_op_stats(op->opnum);
 	}
+	if (!rqstp->rq_usedeferral && status == nfserr_dropit) {
+		dprintk("%s Dropit - send NFS4ERR_DELAY\n", __func__);
+		status = nfserr_jukebox;
+	}
 
 	cstate_free(cstate);
 out:
 	nfsd4_release_compoundargs(args);
+	/* Reset deferral mechanism for RPC deferrals */
+	rqstp->rq_usedeferral = 1;
 	dprintk("nfsv4 compound returned %d\n", ntohl(status));
 	return status;
 }
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 9f9f699..815dd58 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -230,6 +230,7 @@ struct svc_rqst {
 	struct svc_cred		rq_cred;	/* auth info */
 	void *			rq_xprt_ctxt;	/* transport specific context ptr */
 	struct svc_deferred_req*rq_deferred;	/* deferred request we are replaying */
+	int			rq_usedeferral;	/* use deferral */
 
 	size_t			rq_xprt_hlen;	/* xprt header len */
 	struct xdr_buf		rq_arg;
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index fff09a2..45984cb 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -1023,6 +1023,8 @@ svc_process(struct svc_rqst *rqstp)
 	rqstp->rq_res.tail[0].iov_len = 0;
 	/* Will be turned off only in gss privacy case: */
 	rqstp->rq_splice_ok = 1;
+	/* Will be turned off only when NFSv4 Sessions are used */
+	rqstp->rq_usedeferral = 1;
 
 	/* Setup reply header */
 	rqstp->rq_xprt->xpt_ops->xpo_prep_reply_hdr(rqstp);
diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
index 1e66f24..600d091 100644
--- a/net/sunrpc/svc_xprt.c
+++ b/net/sunrpc/svc_xprt.c
@@ -974,7 +974,7 @@ static struct cache_deferred_req *svc_defer(struct cache_req *req)
 	struct svc_rqst *rqstp = container_of(req, struct svc_rqst, rq_chandle);
 	struct svc_deferred_req *dr;
 
-	if (rqstp->rq_arg.page_len)
+	if (rqstp->rq_arg.page_len || !rqstp->rq_usedeferral)
 		return NULL; /* if more than a page, give up FIXME */
 	if (rqstp->rq_deferred) {
 		dr = rqstp->rq_deferred;
-- 
1.6.2.1

[PATCH v2 02/47] nfsd: embed nfsd4_current_state in nfsd4_compoundres

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Remove the allocation of struct nfsd4_compound_state.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   40 +++++++++-------------------------------
 include/linux/nfsd/xdr4.h |    9 +++++----
 2 files changed, 14 insertions(+), 35 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 7839654..c8e1ad0 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -809,29 +809,6 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
 		nfsdstats.nfs4_opcount[opnum]++;
 }
 
-static void cstate_free(struct nfsd4_compound_state *cstate)
-{
-	if (cstate == NULL)
-		return;
-	fh_put(&cstate->current_fh);
-	fh_put(&cstate->save_fh);
-	BUG_ON(cstate->replay_owner);
-	kfree(cstate);
-}
-
-static struct nfsd4_compound_state *cstate_alloc(void)
-{
-	struct nfsd4_compound_state *cstate;
-
-	cstate = kmalloc(sizeof(struct nfsd4_compound_state), GFP_KERNEL);
-	if (cstate == NULL)
-		return NULL;
-	fh_init(&cstate->current_fh, NFS4_FHSIZE);
-	fh_init(&cstate->save_fh, NFS4_FHSIZE);
-	cstate->replay_owner = NULL;
-	return cstate;
-}
-
 typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
 			      void *);
 
@@ -859,12 +836,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 {
 	struct nfsd4_op	*op;
 	struct nfsd4_operation *opdesc;
-	struct nfsd4_compound_state *cstate = NULL;
+	struct nfsd4_compound_state *cstate = &resp->cstate;
 	int		slack_bytes;
 	__be32		status;
 
 	resp->xbuf = &rqstp->rq_res;
-	resp->p = rqstp->rq_res.head[0].iov_base + rqstp->rq_res.head[0].iov_len;
+	resp->p = rqstp->rq_res.head[0].iov_base +
+						rqstp->rq_res.head[0].iov_len;
 	resp->tagp = resp->p;
 	/* reserve space for: taglen, tag, and opcnt */
 	resp->p += 2 + XDR_QUADLEN(args->taglen);
@@ -873,6 +851,9 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	resp->tag = args->tag;
 	resp->opcnt = 0;
 	resp->rqstp = rqstp;
+	resp->cstate.replay_owner = NULL;
+	fh_init(&resp->cstate.current_fh, NFS4_FHSIZE);
+	fh_init(&resp->cstate.save_fh, NFS4_FHSIZE);
 	/* Use the deferral mechanism only for NFSv4.0 compounds */
 	rqstp->rq_usedeferral = (args->minorversion == 0);
 
@@ -883,11 +864,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
 		goto out;
 
-	status = nfserr_resource;
-	cstate = cstate_alloc();
-	if (cstate == NULL)
-		goto out;
-
 	status = nfs_ok;
 	while (!status && resp->opcnt < args->opcnt) {
 		op = &args->ops[resp->opcnt++];
@@ -964,7 +940,9 @@ encode_op:
 		status = nfserr_jukebox;
 	}
 
-	cstate_free(cstate);
+	fh_put(&resp->cstate.current_fh);
+	fh_put(&resp->cstate.save_fh);
+	BUG_ON(resp->cstate.replay_owner);
 out:
 	nfsd4_release_compoundargs(args);
 	/* Reset deferral mechanism for RPC deferrals */
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 27bd3e3..fd15ddc 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -45,9 +45,9 @@
 #define XDR_LEN(n)                     (((n) + 3) & ~3)
 
 struct nfsd4_compound_state {
-	struct svc_fh current_fh;
-	struct svc_fh save_fh;
-	struct nfs4_stateowner *replay_owner;
+	struct svc_fh		current_fh;
+	struct svc_fh		save_fh;
+	struct nfs4_stateowner	*replay_owner;
 };
 
 struct nfsd4_change_info {
@@ -416,7 +416,8 @@ struct nfsd4_compoundres {
 	u32				taglen;
 	char *				tag;
 	u32				opcnt;
-	__be32 *			tagp; /* where to encode tag and  opcount */
+	__be32 *			tagp; /* tag, opcount encode location */
+	struct nfsd4_compound_state	cstate;
 };
 
 #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
-- 
1.6.2.1

Re: [PATCH v2 02/47] nfsd: embed nfsd4_current_state in nfsd4_compoundres

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:30:52AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Remove the allocation of struct nfsd4_compound_state.

Thanks, applied.  (Some minor fixup required as I wanted to apply this
before #1.)

--b.

> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4proc.c        |   40 +++++++++-------------------------------
>  include/linux/nfsd/xdr4.h |    9 +++++----
>  2 files changed, 14 insertions(+), 35 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index 7839654..c8e1ad0 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -809,29 +809,6 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
>  		nfsdstats.nfs4_opcount[opnum]++;
>  }
>  
> -static void cstate_free(struct nfsd4_compound_state *cstate)
> -{
> -	if (cstate == NULL)
> -		return;
> -	fh_put(&cstate->current_fh);
> -	fh_put(&cstate->save_fh);
> -	BUG_ON(cstate->replay_owner);
> -	kfree(cstate);
> -}
> -
> -static struct nfsd4_compound_state *cstate_alloc(void)
> -{
> -	struct nfsd4_compound_state *cstate;
> -
> -	cstate = kmalloc(sizeof(struct nfsd4_compound_state), GFP_KERNEL);
> -	if (cstate == NULL)
> -		return NULL;
> -	fh_init(&cstate->current_fh, NFS4_FHSIZE);
> -	fh_init(&cstate->save_fh, NFS4_FHSIZE);
> -	cstate->replay_owner = NULL;
> -	return cstate;
> -}
> -
>  typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
>  			      void *);
>  
> @@ -859,12 +836,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  {
>  	struct nfsd4_op	*op;
>  	struct nfsd4_operation *opdesc;
> -	struct nfsd4_compound_state *cstate = NULL;
> +	struct nfsd4_compound_state *cstate = &resp->cstate;
>  	int		slack_bytes;
>  	__be32		status;
>  
>  	resp->xbuf = &rqstp->rq_res;
> -	resp->p = rqstp->rq_res.head[0].iov_base + rqstp->rq_res.head[0].iov_len;
> +	resp->p = rqstp->rq_res.head[0].iov_base +
> +						rqstp->rq_res.head[0].iov_len;
>  	resp->tagp = resp->p;
>  	/* reserve space for: taglen, tag, and opcnt */
>  	resp->p += 2 + XDR_QUADLEN(args->taglen);
> @@ -873,6 +851,9 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  	resp->tag = args->tag;
>  	resp->opcnt = 0;
>  	resp->rqstp = rqstp;
> +	resp->cstate.replay_owner = NULL;
> +	fh_init(&resp->cstate.current_fh, NFS4_FHSIZE);
> +	fh_init(&resp->cstate.save_fh, NFS4_FHSIZE);
>  	/* Use the deferral mechanism only for NFSv4.0 compounds */
>  	rqstp->rq_usedeferral = (args->minorversion == 0);
>  
> @@ -883,11 +864,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
>  		goto out;
>  
> -	status = nfserr_resource;
> -	cstate = cstate_alloc();
> -	if (cstate == NULL)
> -		goto out;
> -
>  	status = nfs_ok;
>  	while (!status && resp->opcnt < args->opcnt) {
>  		op = &args->ops[resp->opcnt++];
> @@ -964,7 +940,9 @@ encode_op:
>  		status = nfserr_jukebox;
>  	}
>  
> -	cstate_free(cstate);
> +	fh_put(&resp->cstate.current_fh);
> +	fh_put(&resp->cstate.save_fh);
> +	BUG_ON(resp->cstate.replay_owner);
>  out:
>  	nfsd4_release_compoundargs(args);
>  	/* Reset deferral mechanism for RPC deferrals */
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 27bd3e3..fd15ddc 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -45,9 +45,9 @@
>  #define XDR_LEN(n)                     (((n) + 3) & ~3)
>  
>  struct nfsd4_compound_state {
> -	struct svc_fh current_fh;
> -	struct svc_fh save_fh;
> -	struct nfs4_stateowner *replay_owner;
> +	struct svc_fh		current_fh;
> +	struct svc_fh		save_fh;
> +	struct nfs4_stateowner	*replay_owner;
>  };
>  
>  struct nfsd4_change_info {
> @@ -416,7 +416,8 @@ struct nfsd4_compoundres {
>  	u32				taglen;
>  	char *				tag;
>  	u32				opcnt;
> -	__be32 *			tagp; /* where to encode tag and  opcount */
> +	__be32 *			tagp; /* tag, opcount encode location */
> +	struct nfsd4_compound_state	cstate;
>  };
>  
>  #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
> -- 
> 1.6.2.1
> 

[PATCH v2 03/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

To be used for sessions DRC replay.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfsd/xdr4.h |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index fd15ddc..e309d55 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -48,6 +48,8 @@ struct nfsd4_compound_state {
 	struct svc_fh		current_fh;
 	struct svc_fh		save_fh;
 	struct nfs4_stateowner	*replay_owner;
+	/* For sessions DRC */
+	struct nfsd4_slot	*slot;
 };
 
 struct nfsd4_change_info {
-- 
1.6.2.1

Re: [PATCH v2 03/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:30:57AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> To be used for sessions DRC replay.

It'd be more helpful to have this combined with whatever patch first
defines and/or uses nfsd4_slot.

--b.

> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  include/linux/nfsd/xdr4.h |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index fd15ddc..e309d55 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -48,6 +48,8 @@ struct nfsd4_compound_state {
>  	struct svc_fh		current_fh;
>  	struct svc_fh		save_fh;
>  	struct nfs4_stateowner	*replay_owner;
> +	/* For sessions DRC */
> +	struct nfsd4_slot	*slot;
>  };
>  
>  struct nfsd4_change_info {
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 03/47] nfsd: add a struct nfsd4_slot pointer to struct nfsd4_compound_state

[Benny Halevy]

On Mar. 29, 2009, 23:46 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:30:57AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> To be used for sessions DRC replay.
> 
> It'd be more helpful to have this combined with whatever patch first
> defines and/or uses nfsd4_slot.

Agreed.  I'll move it there.

Benny

> 
> --b.

[PATCH v2 04/47] nfs41: common protocol definitions

[Benny Halevy]

Define all NFSv4.1 common operation and error code constants.

Note that some of the definitions are used by both the nfs41 client
and the server code. This patch is duplicated in the nfs41 and nfsd41
sessions patchset.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfs41: add exchange id flags]
Signed-off-by: Mike Sager <sager@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[removed server-only hunk changing NFSERR_REPLAY_ME]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfs41: add SEQ4_XX to nfs41-common-protocol]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfs41: generic error code update]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfs4.h |  128 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 127 insertions(+), 1 deletions(-)

diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index b912311..a34683e 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -21,6 +21,7 @@
 #define NFS4_FHSIZE		128
 #define NFS4_MAXPATHLEN		PATH_MAX
 #define NFS4_MAXNAMLEN		NAME_MAX
+#define NFS4_MAX_SESSIONID_LEN	16
 
 #define NFS4_ACCESS_READ        0x0001
 #define NFS4_ACCESS_LOOKUP      0x0002
@@ -38,6 +39,7 @@
 #define NFS4_OPEN_RESULT_CONFIRM 0x0002
 #define NFS4_OPEN_RESULT_LOCKTYPE_POSIX 0x0004
 
+#define NFS4_SHARE_ACCESS_MASK	0x000F
 #define NFS4_SHARE_ACCESS_READ	0x0001
 #define NFS4_SHARE_ACCESS_WRITE	0x0002
 #define NFS4_SHARE_ACCESS_BOTH	0x0003
@@ -45,6 +47,19 @@
 #define NFS4_SHARE_DENY_WRITE	0x0002
 #define NFS4_SHARE_DENY_BOTH	0x0003
 
+/* nfs41 */
+#define NFS4_SHARE_WANT_MASK		0xFF00
+#define NFS4_SHARE_WANT_NO_PREFERENCE	0x0000
+#define NFS4_SHARE_WANT_READ_DELEG	0x0100
+#define NFS4_SHARE_WANT_WRITE_DELEG	0x0200
+#define NFS4_SHARE_WANT_ANY_DELEG	0x0300
+#define NFS4_SHARE_WANT_NO_DELEG	0x0400
+#define NFS4_SHARE_WANT_CANCEL		0x0500
+
+#define NFS4_SHARE_WHEN_MASK		0xF0000
+#define NFS4_SHARE_SIGNAL_DELEG_WHEN_RESRC_AVAIL	0x10000
+#define NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED		0x20000
+
 #define NFS4_SET_TO_SERVER_TIME	0
 #define NFS4_SET_TO_CLIENT_TIME	1
 
@@ -88,6 +103,31 @@
 #define NFS4_ACE_GENERIC_EXECUTE              0x001200A0
 #define NFS4_ACE_MASK_ALL                     0x001F01FF
 
+#define EXCHGID4_FLAG_SUPP_MOVED_REFER		0x00000001
+#define EXCHGID4_FLAG_SUPP_MOVED_MIGR		0x00000002
+#define EXCHGID4_FLAG_USE_NON_PNFS		0x00010000
+#define EXCHGID4_FLAG_USE_PNFS_MDS		0x00020000
+#define EXCHGID4_FLAG_USE_PNFS_DS		0x00040000
+#define EXCHGID4_FLAG_UPD_CONFIRMED_REC_A	0x40000000
+#define EXCHGID4_FLAG_CONFIRMED_R		0x80000000
+/*
+ * Since the validity of these bits depends on whether
+ * they're set in the argument or response, have separate
+ * invalid flag masks for arg (_A) and resp (_R).
+ */
+#define EXCHGID4_INVAL_FLAG_MASK_A		0xBFF8FFFC
+#define EXCHGID4_INVAL_FLAG_MASK_R		0x7FF8FFFC
+
+#define SEQ4_STATUS_CB_PATH_DOWN		0x00000001
+#define SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRING	0x00000002
+#define SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRED	0x00000004
+#define SEQ4_STATUS_EXPIRED_ALL_STATE_REVOKED	0x00000008
+#define SEQ4_STATUS_EXPIRED_SOME_STATE_REVOKED	0x00000010
+#define SEQ4_STATUS_ADMIN_STATE_REVOKED		0x00000020
+#define SEQ4_STATUS_RECALLABLE_STATE_REVOKED	0x00000040
+#define SEQ4_STATUS_LEASE_MOVED			0x00000080
+#define SEQ4_STATUS_RESTART_RECLAIM_NEEDED	0x00000100
+
 #define NFS4_MAX_UINT64	(~(u64)0)
 
 enum nfs4_acl_whotype {
@@ -154,6 +194,28 @@ enum nfs_opnum4 {
 	OP_VERIFY = 37,
 	OP_WRITE = 38,
 	OP_RELEASE_LOCKOWNER = 39,
+
+	/* nfs41 */
+	OP_BACKCHANNEL_CTL = 40,
+	OP_BIND_CONN_TO_SESSION = 41,
+	OP_EXCHANGE_ID = 42,
+	OP_CREATE_SESSION = 43,
+	OP_DESTROY_SESSION = 44,
+	OP_FREE_STATEID = 45,
+	OP_GET_DIR_DELEGATION = 46,
+	OP_GETDEVICEINFO = 47,
+	OP_GETDEVICELIST = 48,
+	OP_LAYOUTCOMMIT = 49,
+	OP_LAYOUTGET = 50,
+	OP_LAYOUTRETURN = 51,
+	OP_SECINFO_NO_NAME = 52,
+	OP_SEQUENCE = 53,
+	OP_SET_SSV = 54,
+	OP_TEST_STATEID = 55,
+	OP_WANT_DELEGATION = 56,
+	OP_DESTROY_CLIENTID = 57,
+	OP_RECLAIM_COMPLETE = 58,
+
 	OP_ILLEGAL = 10044,
 };
 
@@ -230,7 +292,48 @@ enum nfsstat4 {
 	NFS4ERR_DEADLOCK = 10045,
 	NFS4ERR_FILE_OPEN = 10046,
 	NFS4ERR_ADMIN_REVOKED = 10047,
-	NFS4ERR_CB_PATH_DOWN = 10048
+	NFS4ERR_CB_PATH_DOWN = 10048,
+
+	/* nfs41 */
+	NFS4ERR_BADIOMODE	= 10049,
+	NFS4ERR_BADLAYOUT	= 10050,
+	NFS4ERR_BAD_SESSION_DIGEST = 10051,
+	NFS4ERR_BADSESSION	= 10052,
+	NFS4ERR_BADSLOT		= 10053,
+	NFS4ERR_COMPLETE_ALREADY = 10054,
+	NFS4ERR_CONN_NOT_BOUND_TO_SESSION = 10055,
+	NFS4ERR_DELEG_ALREADY_WANTED = 10056,
+	NFS4ERR_BACK_CHAN_BUSY	= 10057,	/* backchan reqs outstanding */
+	NFS4ERR_LAYOUTTRYLATER	= 10058,
+	NFS4ERR_LAYOUTUNAVAILABLE = 10059,
+	NFS4ERR_NOMATCHING_LAYOUT = 10060,
+	NFS4ERR_RECALLCONFLICT	= 10061,
+	NFS4ERR_UNKNOWN_LAYOUTTYPE = 10062,
+	NFS4ERR_SEQ_MISORDERED = 10063, 	/* unexpected seq.id in req */
+	NFS4ERR_SEQUENCE_POS	= 10064,	/* [CB_]SEQ. op not 1st op */
+	NFS4ERR_REQ_TOO_BIG	= 10065,	/* request too big */
+	NFS4ERR_REP_TOO_BIG	= 10066,	/* reply too big */
+	NFS4ERR_REP_TOO_BIG_TO_CACHE = 10067,	/* rep. not all cached */
+	NFS4ERR_RETRY_UNCACHED_REP = 10068,	/* retry & rep. uncached */
+	NFS4ERR_UNSAFE_COMPOUND = 10069,	/* retry/recovery too hard */
+	NFS4ERR_TOO_MANY_OPS	= 10070,	/* too many ops in [CB_]COMP */
+	NFS4ERR_OP_NOT_IN_SESSION = 10071,	/* op needs [CB_]SEQ. op */
+	NFS4ERR_HASH_ALG_UNSUPP = 10072,	/* hash alg. not supp. */
+						/* Error 10073 is unused. */
+	NFS4ERR_CLIENTID_BUSY	= 10074,	/* clientid has state */
+	NFS4ERR_PNFS_IO_HOLE	= 10075,	/* IO to _SPARSE file hole */
+	NFS4ERR_SEQ_FALSE_RETRY	= 10076,	/* retry not origional */
+	NFS4ERR_BAD_HIGH_SLOT	= 10077,	/* sequence arg bad */
+	NFS4ERR_DEADSESSION	= 10078,	/* persistent session dead */
+	NFS4ERR_ENCR_ALG_UNSUPP = 10079,	/* SSV alg mismatch */
+	NFS4ERR_PNFS_NO_LAYOUT	= 10080,	/* direct I/O with no layout */
+	NFS4ERR_NOT_ONLY_OP	= 10081,	/* bad compound */
+	NFS4ERR_WRONG_CRED	= 10082,	/* permissions:state change */
+	NFS4ERR_WRONG_TYPE	= 10083,	/* current operation mismatch */
+	NFS4ERR_DIRDELEG_UNAVAIL = 10084,	/* no directory delegation */
+	NFS4ERR_REJECT_DELEG	= 10085,	/* on callback */
+	NFS4ERR_RETURNCONFLICT	= 10086,	/* outstanding layoutreturn */
+	NFS4ERR_DELEG_REVOKED	= 10087,	/* deleg./layout revoked */
 };
 
 /*
@@ -391,6 +494,29 @@ enum {
 	NFSPROC4_CLNT_GETACL,
 	NFSPROC4_CLNT_SETACL,
 	NFSPROC4_CLNT_FS_LOCATIONS,
+
+	/* nfs41 */
+	NFSPROC4_CLNT_EXCHANGE_ID,
+	NFSPROC4_CLNT_CREATE_SESSION,
+	NFSPROC4_CLNT_DESTROY_SESSION,
+	NFSPROC4_CLNT_SEQUENCE,
+	NFSPROC4_CLNT_GET_LEASE_TIME,
+};
+
+/* nfs41 types */
+struct nfs4_sessionid {
+	unsigned char data[NFS4_MAX_SESSIONID_LEN];
+};
+
+/* Create Session Flags */
+#define SESSION4_PERSIST	 0x001
+#define SESSION4_BACK_CHAN 	 0x002
+#define SESSION4_RDMA		 0x004
+
+enum state_protect_how4 {
+	SP4_NONE	= 0,
+	SP4_MACH_CRED	= 1,
+	SP4_SSV		= 2
 };
 
 #endif
-- 
1.6.2.1

Re: [PATCH v2 04/47] nfs41: common protocol definitions

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:31:01AM +0300, Benny Halevy wrote:
> Define all NFSv4.1 common operation and error code constants.
> 
> Note that some of the definitions are used by both the nfs41 client
> and the server code. This patch is duplicated in the nfs41 and nfsd41
> sessions patchset.
> 
> Signed-off-by: Andy Adamson<andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfs41: add exchange id flags]
> Signed-off-by: Mike Sager <sager@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [removed server-only hunk changing NFSERR_REPLAY_ME]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfs41: add SEQ4_XX to nfs41-common-protocol]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfs41: generic error code update]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  include/linux/nfs4.h |  128 +++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 127 insertions(+), 1 deletions(-)
> 
> diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
> index b912311..a34683e 100644
> --- a/include/linux/nfs4.h
> +++ b/include/linux/nfs4.h
> @@ -21,6 +21,7 @@
>  #define NFS4_FHSIZE		128
>  #define NFS4_MAXPATHLEN		PATH_MAX
>  #define NFS4_MAXNAMLEN		NAME_MAX
> +#define NFS4_MAX_SESSIONID_LEN	16
>  
>  #define NFS4_ACCESS_READ        0x0001
>  #define NFS4_ACCESS_LOOKUP      0x0002
> @@ -38,6 +39,7 @@
>  #define NFS4_OPEN_RESULT_CONFIRM 0x0002
>  #define NFS4_OPEN_RESULT_LOCKTYPE_POSIX 0x0004
>  
> +#define NFS4_SHARE_ACCESS_MASK	0x000F
>  #define NFS4_SHARE_ACCESS_READ	0x0001
>  #define NFS4_SHARE_ACCESS_WRITE	0x0002
>  #define NFS4_SHARE_ACCESS_BOTH	0x0003
> @@ -45,6 +47,19 @@
>  #define NFS4_SHARE_DENY_WRITE	0x0002
>  #define NFS4_SHARE_DENY_BOTH	0x0003
>  
> +/* nfs41 */
> +#define NFS4_SHARE_WANT_MASK		0xFF00
> +#define NFS4_SHARE_WANT_NO_PREFERENCE	0x0000
> +#define NFS4_SHARE_WANT_READ_DELEG	0x0100
> +#define NFS4_SHARE_WANT_WRITE_DELEG	0x0200
> +#define NFS4_SHARE_WANT_ANY_DELEG	0x0300
> +#define NFS4_SHARE_WANT_NO_DELEG	0x0400
> +#define NFS4_SHARE_WANT_CANCEL		0x0500
> +
> +#define NFS4_SHARE_WHEN_MASK		0xF0000
> +#define NFS4_SHARE_SIGNAL_DELEG_WHEN_RESRC_AVAIL	0x10000
> +#define NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED		0x20000
> +
>  #define NFS4_SET_TO_SERVER_TIME	0
>  #define NFS4_SET_TO_CLIENT_TIME	1
>  
> @@ -88,6 +103,31 @@
>  #define NFS4_ACE_GENERIC_EXECUTE              0x001200A0
>  #define NFS4_ACE_MASK_ALL                     0x001F01FF
>  
> +#define EXCHGID4_FLAG_SUPP_MOVED_REFER		0x00000001
> +#define EXCHGID4_FLAG_SUPP_MOVED_MIGR		0x00000002
> +#define EXCHGID4_FLAG_USE_NON_PNFS		0x00010000
> +#define EXCHGID4_FLAG_USE_PNFS_MDS		0x00020000
> +#define EXCHGID4_FLAG_USE_PNFS_DS		0x00040000
> +#define EXCHGID4_FLAG_UPD_CONFIRMED_REC_A	0x40000000
> +#define EXCHGID4_FLAG_CONFIRMED_R		0x80000000
> +/*
> + * Since the validity of these bits depends on whether
> + * they're set in the argument or response, have separate
> + * invalid flag masks for arg (_A) and resp (_R).
> + */
> +#define EXCHGID4_INVAL_FLAG_MASK_A		0xBFF8FFFC
> +#define EXCHGID4_INVAL_FLAG_MASK_R		0x7FF8FFFC

Would it be less confusing just to use !EXCHGID_FLAG_MASK_A and
!EXCHGID_FLAG_MASK_R everywhere?

--b.

> +
> +#define SEQ4_STATUS_CB_PATH_DOWN		0x00000001
> +#define SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRING	0x00000002
> +#define SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRED	0x00000004
> +#define SEQ4_STATUS_EXPIRED_ALL_STATE_REVOKED	0x00000008
> +#define SEQ4_STATUS_EXPIRED_SOME_STATE_REVOKED	0x00000010
> +#define SEQ4_STATUS_ADMIN_STATE_REVOKED		0x00000020
> +#define SEQ4_STATUS_RECALLABLE_STATE_REVOKED	0x00000040
> +#define SEQ4_STATUS_LEASE_MOVED			0x00000080
> +#define SEQ4_STATUS_RESTART_RECLAIM_NEEDED	0x00000100
> +
>  #define NFS4_MAX_UINT64	(~(u64)0)
>  
>  enum nfs4_acl_whotype {
> @@ -154,6 +194,28 @@ enum nfs_opnum4 {
>  	OP_VERIFY = 37,
>  	OP_WRITE = 38,
>  	OP_RELEASE_LOCKOWNER = 39,
> +
> +	/* nfs41 */
> +	OP_BACKCHANNEL_CTL = 40,
> +	OP_BIND_CONN_TO_SESSION = 41,
> +	OP_EXCHANGE_ID = 42,
> +	OP_CREATE_SESSION = 43,
> +	OP_DESTROY_SESSION = 44,
> +	OP_FREE_STATEID = 45,
> +	OP_GET_DIR_DELEGATION = 46,
> +	OP_GETDEVICEINFO = 47,
> +	OP_GETDEVICELIST = 48,
> +	OP_LAYOUTCOMMIT = 49,
> +	OP_LAYOUTGET = 50,
> +	OP_LAYOUTRETURN = 51,
> +	OP_SECINFO_NO_NAME = 52,
> +	OP_SEQUENCE = 53,
> +	OP_SET_SSV = 54,
> +	OP_TEST_STATEID = 55,
> +	OP_WANT_DELEGATION = 56,
> +	OP_DESTROY_CLIENTID = 57,
> +	OP_RECLAIM_COMPLETE = 58,
> +
>  	OP_ILLEGAL = 10044,
>  };
>  
> @@ -230,7 +292,48 @@ enum nfsstat4 {
>  	NFS4ERR_DEADLOCK = 10045,
>  	NFS4ERR_FILE_OPEN = 10046,
>  	NFS4ERR_ADMIN_REVOKED = 10047,
> -	NFS4ERR_CB_PATH_DOWN = 10048
> +	NFS4ERR_CB_PATH_DOWN = 10048,
> +
> +	/* nfs41 */
> +	NFS4ERR_BADIOMODE	= 10049,
> +	NFS4ERR_BADLAYOUT	= 10050,
> +	NFS4ERR_BAD_SESSION_DIGEST = 10051,
> +	NFS4ERR_BADSESSION	= 10052,
> +	NFS4ERR_BADSLOT		= 10053,
> +	NFS4ERR_COMPLETE_ALREADY = 10054,
> +	NFS4ERR_CONN_NOT_BOUND_TO_SESSION = 10055,
> +	NFS4ERR_DELEG_ALREADY_WANTED = 10056,
> +	NFS4ERR_BACK_CHAN_BUSY	= 10057,	/* backchan reqs outstanding */
> +	NFS4ERR_LAYOUTTRYLATER	= 10058,
> +	NFS4ERR_LAYOUTUNAVAILABLE = 10059,
> +	NFS4ERR_NOMATCHING_LAYOUT = 10060,
> +	NFS4ERR_RECALLCONFLICT	= 10061,
> +	NFS4ERR_UNKNOWN_LAYOUTTYPE = 10062,
> +	NFS4ERR_SEQ_MISORDERED = 10063, 	/* unexpected seq.id in req */
> +	NFS4ERR_SEQUENCE_POS	= 10064,	/* [CB_]SEQ. op not 1st op */
> +	NFS4ERR_REQ_TOO_BIG	= 10065,	/* request too big */
> +	NFS4ERR_REP_TOO_BIG	= 10066,	/* reply too big */
> +	NFS4ERR_REP_TOO_BIG_TO_CACHE = 10067,	/* rep. not all cached */
> +	NFS4ERR_RETRY_UNCACHED_REP = 10068,	/* retry & rep. uncached */
> +	NFS4ERR_UNSAFE_COMPOUND = 10069,	/* retry/recovery too hard */
> +	NFS4ERR_TOO_MANY_OPS	= 10070,	/* too many ops in [CB_]COMP */
> +	NFS4ERR_OP_NOT_IN_SESSION = 10071,	/* op needs [CB_]SEQ. op */
> +	NFS4ERR_HASH_ALG_UNSUPP = 10072,	/* hash alg. not supp. */
> +						/* Error 10073 is unused. */
> +	NFS4ERR_CLIENTID_BUSY	= 10074,	/* clientid has state */
> +	NFS4ERR_PNFS_IO_HOLE	= 10075,	/* IO to _SPARSE file hole */
> +	NFS4ERR_SEQ_FALSE_RETRY	= 10076,	/* retry not origional */
> +	NFS4ERR_BAD_HIGH_SLOT	= 10077,	/* sequence arg bad */
> +	NFS4ERR_DEADSESSION	= 10078,	/* persistent session dead */
> +	NFS4ERR_ENCR_ALG_UNSUPP = 10079,	/* SSV alg mismatch */
> +	NFS4ERR_PNFS_NO_LAYOUT	= 10080,	/* direct I/O with no layout */
> +	NFS4ERR_NOT_ONLY_OP	= 10081,	/* bad compound */
> +	NFS4ERR_WRONG_CRED	= 10082,	/* permissions:state change */
> +	NFS4ERR_WRONG_TYPE	= 10083,	/* current operation mismatch */
> +	NFS4ERR_DIRDELEG_UNAVAIL = 10084,	/* no directory delegation */
> +	NFS4ERR_REJECT_DELEG	= 10085,	/* on callback */
> +	NFS4ERR_RETURNCONFLICT	= 10086,	/* outstanding layoutreturn */
> +	NFS4ERR_DELEG_REVOKED	= 10087,	/* deleg./layout revoked */
>  };
>  
>  /*
> @@ -391,6 +494,29 @@ enum {
>  	NFSPROC4_CLNT_GETACL,
>  	NFSPROC4_CLNT_SETACL,
>  	NFSPROC4_CLNT_FS_LOCATIONS,
> +
> +	/* nfs41 */
> +	NFSPROC4_CLNT_EXCHANGE_ID,
> +	NFSPROC4_CLNT_CREATE_SESSION,
> +	NFSPROC4_CLNT_DESTROY_SESSION,
> +	NFSPROC4_CLNT_SEQUENCE,
> +	NFSPROC4_CLNT_GET_LEASE_TIME,
> +};
> +
> +/* nfs41 types */
> +struct nfs4_sessionid {
> +	unsigned char data[NFS4_MAX_SESSIONID_LEN];
> +};
> +
> +/* Create Session Flags */
> +#define SESSION4_PERSIST	 0x001
> +#define SESSION4_BACK_CHAN 	 0x002
> +#define SESSION4_RDMA		 0x004
> +
> +enum state_protect_how4 {
> +	SP4_NONE	= 0,
> +	SP4_MACH_CRED	= 1,
> +	SP4_SSV		= 2
>  };
>  
>  #endif
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 04/47] nfs41: common protocol definitions

[Benny Halevy]

On Mar. 31, 2009, 0:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:31:01AM +0300, Benny Halevy wrote:
>> Define all NFSv4.1 common operation and error code constants.
>>
>> Note that some of the definitions are used by both the nfs41 client
>> and the server code. This patch is duplicated in the nfs41 and nfsd41
>> sessions patchset.
>>
>> Signed-off-by: Andy Adamson<andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfs41: add exchange id flags]
>> Signed-off-by: Mike Sager <sager@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [removed server-only hunk changing NFSERR_REPLAY_ME]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfs41: add SEQ4_XX to nfs41-common-protocol]
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfs41: generic error code update]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  include/linux/nfs4.h |  128 +++++++++++++++++++++++++++++++++++++++++++++++++-
>>  1 files changed, 127 insertions(+), 1 deletions(-)
>>
>> diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
>> index b912311..a34683e 100644
>> --- a/include/linux/nfs4.h
>> +++ b/include/linux/nfs4.h
>> @@ -21,6 +21,7 @@
>>  #define NFS4_FHSIZE		128
>>  #define NFS4_MAXPATHLEN		PATH_MAX
>>  #define NFS4_MAXNAMLEN		NAME_MAX
>> +#define NFS4_MAX_SESSIONID_LEN	16
>>  
>>  #define NFS4_ACCESS_READ        0x0001
>>  #define NFS4_ACCESS_LOOKUP      0x0002
>> @@ -38,6 +39,7 @@
>>  #define NFS4_OPEN_RESULT_CONFIRM 0x0002
>>  #define NFS4_OPEN_RESULT_LOCKTYPE_POSIX 0x0004
>>  
>> +#define NFS4_SHARE_ACCESS_MASK	0x000F
>>  #define NFS4_SHARE_ACCESS_READ	0x0001
>>  #define NFS4_SHARE_ACCESS_WRITE	0x0002
>>  #define NFS4_SHARE_ACCESS_BOTH	0x0003
>> @@ -45,6 +47,19 @@
>>  #define NFS4_SHARE_DENY_WRITE	0x0002
>>  #define NFS4_SHARE_DENY_BOTH	0x0003
>>  
>> +/* nfs41 */
>> +#define NFS4_SHARE_WANT_MASK		0xFF00
>> +#define NFS4_SHARE_WANT_NO_PREFERENCE	0x0000
>> +#define NFS4_SHARE_WANT_READ_DELEG	0x0100
>> +#define NFS4_SHARE_WANT_WRITE_DELEG	0x0200
>> +#define NFS4_SHARE_WANT_ANY_DELEG	0x0300
>> +#define NFS4_SHARE_WANT_NO_DELEG	0x0400
>> +#define NFS4_SHARE_WANT_CANCEL		0x0500
>> +
>> +#define NFS4_SHARE_WHEN_MASK		0xF0000
>> +#define NFS4_SHARE_SIGNAL_DELEG_WHEN_RESRC_AVAIL	0x10000
>> +#define NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED		0x20000
>> +
>>  #define NFS4_SET_TO_SERVER_TIME	0
>>  #define NFS4_SET_TO_CLIENT_TIME	1
>>  
>> @@ -88,6 +103,31 @@
>>  #define NFS4_ACE_GENERIC_EXECUTE              0x001200A0
>>  #define NFS4_ACE_MASK_ALL                     0x001F01FF
>>  
>> +#define EXCHGID4_FLAG_SUPP_MOVED_REFER		0x00000001
>> +#define EXCHGID4_FLAG_SUPP_MOVED_MIGR		0x00000002
>> +#define EXCHGID4_FLAG_USE_NON_PNFS		0x00010000
>> +#define EXCHGID4_FLAG_USE_PNFS_MDS		0x00020000
>> +#define EXCHGID4_FLAG_USE_PNFS_DS		0x00040000
>> +#define EXCHGID4_FLAG_UPD_CONFIRMED_REC_A	0x40000000
>> +#define EXCHGID4_FLAG_CONFIRMED_R		0x80000000
>> +/*
>> + * Since the validity of these bits depends on whether
>> + * they're set in the argument or response, have separate
>> + * invalid flag masks for arg (_A) and resp (_R).
>> + */
>> +#define EXCHGID4_INVAL_FLAG_MASK_A		0xBFF8FFFC
>> +#define EXCHGID4_INVAL_FLAG_MASK_R		0x7FF8FFFC
> 
> Would it be less confusing just to use !EXCHGID_FLAG_MASK_A and
> !EXCHGID_FLAG_MASK_R everywhere?

Fine with me.
Mike, this code came from you, what do you think?
Please Ack.  Thanks,

Benny

> 
> --b.
> 

[PATCH v2 05/47] nfsd41: change NFSERR_REPLAY_ME

[Benny Halevy]

so not to overlap nfs41 error codes.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfs.h |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/include/linux/nfs.h b/include/linux/nfs.h
index 54af92c..1510a64 100644
--- a/include/linux/nfs.h
+++ b/include/linux/nfs.h
@@ -109,7 +109,8 @@
 	NFSERR_FILE_OPEN = 10046,      /*       v4 */
 	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
 	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
-	NFSERR_REPLAY_ME = 10049	/*       v4 */
+
+	NFSERR_REPLAY_ME = 11001,	/* linux internal */
 };
 
 /* NFSv2 file types - beware, these are not the same in NFSv3 */
-- 
1.6.2.1

[PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment

At the moment we're expecting folks trying out nfs41 to
actively participate in the development process by helping us
debug issues and ideally send patches to fix problems.

[trailing space fixed]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/Kconfig |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/Kconfig b/fs/nfsd/Kconfig
index 503b9da..1f2f893 100644
--- a/fs/nfsd/Kconfig
+++ b/fs/nfsd/Kconfig
@@ -79,3 +79,12 @@ config NFSD_V4
 	  available from http://linux-nfs.org/.
 
 	  If unsure, say N.
+
+config NFSD_V4_1
+	bool "NFS server support for NFSv4.1 (DEVELOPER ONLY)"
+	depends on NFSD_V4 && EXPERIMENTAL
+	help
+	  This option enables support for minor version 1 of the NFSv4 protocol
+	  (draft-ietf-nfsv4-minorversion1) in the kernel's NFS server.
+
+	  Unless you're an NFS developer, say N.
-- 
1.6.2.1

Re: [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment

Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
will want to build a kernel with v4.0 but not v4.1?

(And: do we have an interface that allows turning off 4.1 at run-time?
That's more important than the config option.)

--b.

> 
> At the moment we're expecting folks trying out nfs41 to
> actively participate in the development process by helping us
> debug issues and ideally send patches to fix problems.
> 
> [trailing space fixed]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/Kconfig |    9 +++++++++
>  1 files changed, 9 insertions(+), 0 deletions(-)
> 
> diff --git a/fs/nfsd/Kconfig b/fs/nfsd/Kconfig
> index 503b9da..1f2f893 100644
> --- a/fs/nfsd/Kconfig
> +++ b/fs/nfsd/Kconfig
> @@ -79,3 +79,12 @@ config NFSD_V4
>  	  available from http://linux-nfs.org/.
>  
>  	  If unsure, say N.
> +
> +config NFSD_V4_1
> +	bool "NFS server support for NFSv4.1 (DEVELOPER ONLY)"
> +	depends on NFSD_V4 && EXPERIMENTAL
> +	help
> +	  This option enables support for minor version 1 of the NFSv4 protocol
> +	  (draft-ietf-nfsv4-minorversion1) in the kernel's NFS server.
> +
> +	  Unless you're an NFS developer, say N.
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> 
> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> will want to build a kernel with v4.0 but not v4.1?

That's a good question.  I'd love to get rid of it
and it seems like like distros, at least RH are going to have it
configured-in anyway.

If the main reason to turn 4.1 support off is bugs affecting 4.0
then I'd much rather fix these bugs rather than hide them.

> 
> (And: do we have an interface that allows turning off 4.1 at run-time?
> That's more important than the config option.)

No, it's still on our todo list.  We haven't thought this completely
through, though.  Where would be the best place to implement that?
Should this be an export option or an nfsd tunable?

Benny

> 
> --b.
> 
>> At the moment we're expecting folks trying out nfs41 to
>> actively participate in the development process by helping us
>> debug issues and ideally send patches to fix problems.
>>
>> [trailing space fixed]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/Kconfig |    9 +++++++++
>>  1 files changed, 9 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/nfsd/Kconfig b/fs/nfsd/Kconfig
>> index 503b9da..1f2f893 100644
>> --- a/fs/nfsd/Kconfig
>> +++ b/fs/nfsd/Kconfig
>> @@ -79,3 +79,12 @@ config NFSD_V4
>>  	  available from http://linux-nfs.org/.
>>  
>>  	  If unsure, say N.
>> +
>> +config NFSD_V4_1
>> +	bool "NFS server support for NFSv4.1 (DEVELOPER ONLY)"
>> +	depends on NFSD_V4 && EXPERIMENTAL
>> +	help
>> +	  This option enables support for minor version 1 of the NFSv4 protocol
>> +	  (draft-ietf-nfsv4-minorversion1) in the kernel's NFS server.
>> +
>> +	  Unless you're an NFS developer, say N.
>> -- 
>> 1.6.2.1
>>

Re: [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> >> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> >> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> > 
> > Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> > will want to build a kernel with v4.0 but not v4.1?
> 
> That's a good question.  I'd love to get rid of it
> and it seems like like distros, at least RH are going to have it
> configured-in anyway.
> 
> If the main reason to turn 4.1 support off is bugs affecting 4.0
> then I'd much rather fix these bugs rather than hide them.
> 
> > 
> > (And: do we have an interface that allows turning off 4.1 at run-time?
> > That's more important than the config option.)
> 
> No, it's still on our todo list.  We haven't thought this completely
> through, though.  Where would be the best place to implement that?
> Should this be an export option or an nfsd tunable?

Imitating (or extending, if possible) nfsd/versions would be one way.
See fs/nfsd/nfsctl.c:write_versions.  Cc'ing Steved, as I think that was
originally his work.

--b.

Re: [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
>>> will want to build a kernel with v4.0 but not v4.1?
>> That's a good question.  I'd love to get rid of it
>> and it seems like like distros, at least RH are going to have it
>> configured-in anyway.
>>
>> If the main reason to turn 4.1 support off is bugs affecting 4.0
>> then I'd much rather fix these bugs rather than hide them.
>>
>>> (And: do we have an interface that allows turning off 4.1 at run-time?
>>> That's more important than the config option.)
>> No, it's still on our todo list.  We haven't thought this completely
>> through, though.  Where would be the best place to implement that?
>> Should this be an export option or an nfsd tunable?
> 
> Imitating (or extending, if possible) nfsd/versions would be one way.
> See fs/nfsd/nfsctl.c:write_versions.  Cc'ing Steved, as I think that was
> originally his work.
> 
> --b.


Sounds great.
Here's a crude untested patch.  Does that what you mean?

Benny

git diff --stat -p
 fs/nfsd/nfs4proc.c        |    2 +-
 fs/nfsd/nfsctl.c          |   31 ++++++++++++++++++++++++++-----
 fs/nfsd/nfssvc.c          |   24 ++++++++++++++++++++++++
 include/linux/nfsd/nfsd.h |    2 ++
 4 files changed, 53 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index a393d38..7619970 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -943,7 +943,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	 * According to RFC3010, this takes precedence over all other errors.
 	 */
 	status = nfserr_minor_vers_mismatch;
-	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
+	if (args->minorversion > nfsd_supported_minorversion)
 		goto out;
 
 	if (!nfs41_op_ordering_ok(args)) {
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 4d63010..5fe1930 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -792,8 +792,9 @@ out_free:
 static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 {
 	char *mesg = buf;
-	char *vers, sign;
+	char *vers, *minorp, sign;
 	int len, num;
+	unsigned minor;
 	ssize_t tlen = 0;
 	char *sep;
 
@@ -814,9 +815,20 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 		do {
 			sign = *vers;
 			if (sign == '+' || sign == '-')
-				num = simple_strtol((vers+1), NULL, 0);
+				num = simple_strtol((vers+1), &minorp, 0);
 			else
-				num = simple_strtol(vers, NULL, 0);
+				num = simple_strtol(vers, &minorp, 0);
+			if (*minorp == '.') {
+				if (num < 4)
+					return -EINVAL;
+				minor = simple_strtoul(minorp+1, NULL, 0);
+				if (minor == 0)
+					return -EINVAL;
+				if (nfsd_minorversion(minor, sign == '-' ?
+						     NFSD_CLEAR : NFSD_SET) < 0)
+					return -EINVAL;
+				goto next;
+			}
 			switch(num) {
 			case 2:
 			case 3:
@@ -826,6 +838,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 			default:
 				return -EINVAL;
 			}
+		next:
 			vers += len + 1;
 			tlen += len;
 		} while ((len = qword_get(&mesg, vers, size)) > 0);
@@ -839,12 +852,20 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 	sep = "";
 	for (num=2 ; num <= 4 ; num++)
 		if (nfsd_vers(num, NFSD_AVAIL)) {
-			len += sprintf(buf+len, "%s%c%d", sep,
+			len += snprintf(buf+len, size-len, "%s%c%d", sep,
 				       nfsd_vers(num, NFSD_TEST)?'+':'-',
 				       num);
 			sep = " ";
 		}
-	len += sprintf(buf+len, "\n");
+	if (nfsd_vers(4, NFSD_AVAIL)) {
+		for (minor = 1; minor <= NFSD_SUPPORTED_MINOR_VERSION; minor++)
+			if (nfsd_minorversion(minor, NFSD_AVAIL)) {
+				len += snprintf(buf+len, size-len, " %c4.%u",
+					      nfsd_vers(num, NFSD_TEST)?'+':'-',
+					      minor);
+			}
+	}
+	len += snprintf(buf+len, size-len, "\n");
 	return len;
 }
 
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 6ec29c7..5f4cecb 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -121,6 +121,8 @@ struct svc_program		nfsd_program = {
 
 };
 
+u32 nfsd_supported_minorversion = NFSD_SUPPORTED_MINOR_VERSION;
+
 int nfsd_vers(int vers, enum vers_op change)
 {
 	if (vers < NFSD_MINVERS || vers >= NFSD_NRVERS)
@@ -147,6 +149,28 @@ int nfsd_vers(int vers, enum vers_op change)
 	}
 	return 0;
 }
+
+int nfsd_minorversion(u32 minorversion, enum vers_op change)
+{
+	if (minorversion > NFSD_SUPPORTED_MINOR_VERSION)
+		return -1;
+	switch(change) {
+	case NFSD_SET:
+		nfsd_supported_minorversion = minorversion;
+		break;
+	case NFSD_CLEAR:
+		if (minorversion == 0)
+			return -1;
+		nfsd_supported_minorversion = minorversion - 1;
+		break;
+	case NFSD_TEST:
+		return minorversion <= nfsd_supported_minorversion;
+	case NFSD_AVAIL:
+		return minorversion <= NFSD_SUPPORTED_MINOR_VERSION;
+	}
+	return 0;
+}
+
 /*
  * Maximum number of nfsd processes
  */
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index b9e6682..27b9cf5 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -57,6 +57,7 @@ typedef int (*nfsd_dirop_t)(struct inode *, struct dentry *, int, int);
 extern struct svc_program	nfsd_program;
 extern struct svc_version	nfsd_version2, nfsd_version3,
 				nfsd_version4;
+extern u32			nfsd_supported_minorversion;
 extern struct mutex		nfsd_mutex;
 extern struct svc_serv		*nfsd_serv;
 
@@ -153,6 +154,7 @@ int nfsd_set_posix_acl(struct svc_fh *, int, struct posix_acl *);
 
 enum vers_op {NFSD_SET, NFSD_CLEAR, NFSD_TEST, NFSD_AVAIL };
 int nfsd_vers(int vers, enum vers_op change);
+int nfsd_minorversion(u32 minorversion, enum vers_op change);
 void nfsd_reset_versions(void);
 int nfsd_create_serv(void);
 

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
>>>> will want to build a kernel with v4.0 but not v4.1?
>>> That's a good question.  I'd love to get rid of it
>>> and it seems like like distros, at least RH are going to have it
>>> configured-in anyway.
>>>
>>> If the main reason to turn 4.1 support off is bugs affecting 4.0
>>> then I'd much rather fix these bugs rather than hide them.
>>>
>>>> (And: do we have an interface that allows turning off 4.1 at run-time?
>>>> That's more important than the config option.)
>>> No, it's still on our todo list.  We haven't thought this completely
>>> through, though.  Where would be the best place to implement that?
>>> Should this be an export option or an nfsd tunable?
>> Imitating (or extending, if possible) nfsd/versions would be one way.
>> See fs/nfsd/nfsctl.c:write_versions.  Cc'ing Steved, as I think that was
>> originally his work.
>>
>> --b.
> 
> 
> Sounds great.
> Here's a crude untested patch.  Does that what you mean?
> 

The following works better.

But I still see a problem with it.
This all works nicely when the nfs service is stopped (on Fedora 9)
but when I restart it all version return to their defaults.
I'm probably doing something wrong...

Benny

git diff --stat -p
 fs/nfsd/nfs4proc.c        |    2 +-
 fs/nfsd/nfsctl.c          |   26 +++++++++++++++++++++++---
 fs/nfsd/nfssvc.c          |   24 ++++++++++++++++++++++++
 include/linux/nfsd/nfsd.h |    2 ++
 4 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index a393d38..7619970 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -943,7 +943,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	 * According to RFC3010, this takes precedence over all other errors.
 	 */
 	status = nfserr_minor_vers_mismatch;
-	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
+	if (args->minorversion > nfsd_supported_minorversion)
 		goto out;
 
 	if (!nfs41_op_ordering_ok(args)) {
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 4d63010..af16849 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -792,8 +792,9 @@ out_free:
 static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 {
 	char *mesg = buf;
-	char *vers, sign;
+	char *vers, *minorp, sign;
 	int len, num;
+	unsigned minor;
 	ssize_t tlen = 0;
 	char *sep;
 
@@ -814,9 +815,20 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 		do {
 			sign = *vers;
 			if (sign == '+' || sign == '-')
-				num = simple_strtol((vers+1), NULL, 0);
+				num = simple_strtol((vers+1), &minorp, 0);
 			else
-				num = simple_strtol(vers, NULL, 0);
+				num = simple_strtol(vers, &minorp, 0);
+			if (*minorp == '.') {
+				if (num < 4)
+					return -EINVAL;
+				minor = simple_strtoul(minorp+1, NULL, 0);
+				if (minor == 0)
+					return -EINVAL;
+				if (nfsd_minorversion(minor, sign == '-' ?
+						     NFSD_CLEAR : NFSD_SET) < 0)
+					return -EINVAL;
+				goto next;
+			}
 			switch(num) {
 			case 2:
 			case 3:
@@ -826,6 +838,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 			default:
 				return -EINVAL;
 			}
+		next:
 			vers += len + 1;
 			tlen += len;
 		} while ((len = qword_get(&mesg, vers, size)) > 0);
@@ -844,6 +857,13 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
 				       num);
 			sep = " ";
 		}
+	if (nfsd_vers(4, NFSD_AVAIL))
+		for (minor = 1; minor <= NFSD_SUPPORTED_MINOR_VERSION; minor++)
+			len += sprintf(buf+len, " %c4.%u",
+					(nfsd_vers(4, NFSD_TEST) &&
+					 nfsd_minorversion(minor, NFSD_TEST)) ?
+						'+' : '-',
+					minor);
 	len += sprintf(buf+len, "\n");
 	return len;
 }
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 6ec29c7..5f4cecb 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -121,6 +121,8 @@ struct svc_program		nfsd_program = {
 
 };
 
+u32 nfsd_supported_minorversion = NFSD_SUPPORTED_MINOR_VERSION;
+
 int nfsd_vers(int vers, enum vers_op change)
 {
 	if (vers < NFSD_MINVERS || vers >= NFSD_NRVERS)
@@ -147,6 +149,28 @@ int nfsd_vers(int vers, enum vers_op change)
 	}
 	return 0;
 }
+
+int nfsd_minorversion(u32 minorversion, enum vers_op change)
+{
+	if (minorversion > NFSD_SUPPORTED_MINOR_VERSION)
+		return -1;
+	switch(change) {
+	case NFSD_SET:
+		nfsd_supported_minorversion = minorversion;
+		break;
+	case NFSD_CLEAR:
+		if (minorversion == 0)
+			return -1;
+		nfsd_supported_minorversion = minorversion - 1;
+		break;
+	case NFSD_TEST:
+		return minorversion <= nfsd_supported_minorversion;
+	case NFSD_AVAIL:
+		return minorversion <= NFSD_SUPPORTED_MINOR_VERSION;
+	}
+	return 0;
+}
+
 /*
  * Maximum number of nfsd processes
  */
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index b9e6682..27b9cf5 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -57,6 +57,7 @@ typedef int (*nfsd_dirop_t)(struct inode *, struct dentry *, int, int);
 extern struct svc_program	nfsd_program;
 extern struct svc_version	nfsd_version2, nfsd_version3,
 				nfsd_version4;
+extern u32			nfsd_supported_minorversion;
 extern struct mutex		nfsd_mutex;
 extern struct svc_serv		*nfsd_serv;
 
@@ -153,6 +154,7 @@ int nfsd_set_posix_acl(struct svc_fh *, int, struct posix_acl *);
 
 enum vers_op {NFSD_SET, NFSD_CLEAR, NFSD_TEST, NFSD_AVAIL };
 int nfsd_vers(int vers, enum vers_op change);
+int nfsd_minorversion(u32 minorversion, enum vers_op change);
 void nfsd_reset_versions(void);
 int nfsd_create_serv(void);
 

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
>>>>> will want to build a kernel with v4.0 but not v4.1?

Bruce,  with the patch below in place, would it be reasonable to
remove CONFIG_NFSD_V4_1?

Benny

>>>> That's a good question.  I'd love to get rid of it
>>>> and it seems like like distros, at least RH are going to have it
>>>> configured-in anyway.
>>>>
>>>> If the main reason to turn 4.1 support off is bugs affecting 4.0
>>>> then I'd much rather fix these bugs rather than hide them.
>>>>
>>>>> (And: do we have an interface that allows turning off 4.1 at run-time?
>>>>> That's more important than the config option.)
>>>> No, it's still on our todo list.  We haven't thought this completely
>>>> through, though.  Where would be the best place to implement that?
>>>> Should this be an export option or an nfsd tunable?
>>> Imitating (or extending, if possible) nfsd/versions would be one way.
>>> See fs/nfsd/nfsctl.c:write_versions.  Cc'ing Steved, as I think that was
>>> originally his work.
>>>
>>> --b.
>>
>> Sounds great.
>> Here's a crude untested patch.  Does that what you mean?
>>
> 
> The following works better.
> 
> But I still see a problem with it.
> This all works nicely when the nfs service is stopped (on Fedora 9)
> but when I restart it all version return to their defaults.
> I'm probably doing something wrong...
> 
> Benny
> 
> git diff --stat -p
>  fs/nfsd/nfs4proc.c        |    2 +-
>  fs/nfsd/nfsctl.c          |   26 +++++++++++++++++++++++---
>  fs/nfsd/nfssvc.c          |   24 ++++++++++++++++++++++++
>  include/linux/nfsd/nfsd.h |    2 ++
>  4 files changed, 50 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index a393d38..7619970 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -943,7 +943,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  	 * According to RFC3010, this takes precedence over all other errors.
>  	 */
>  	status = nfserr_minor_vers_mismatch;
> -	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
> +	if (args->minorversion > nfsd_supported_minorversion)
>  		goto out;
>  
>  	if (!nfs41_op_ordering_ok(args)) {
> diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
> index 4d63010..af16849 100644
> --- a/fs/nfsd/nfsctl.c
> +++ b/fs/nfsd/nfsctl.c
> @@ -792,8 +792,9 @@ out_free:
>  static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>  {
>  	char *mesg = buf;
> -	char *vers, sign;
> +	char *vers, *minorp, sign;
>  	int len, num;
> +	unsigned minor;
>  	ssize_t tlen = 0;
>  	char *sep;
>  
> @@ -814,9 +815,20 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>  		do {
>  			sign = *vers;
>  			if (sign == '+' || sign == '-')
> -				num = simple_strtol((vers+1), NULL, 0);
> +				num = simple_strtol((vers+1), &minorp, 0);
>  			else
> -				num = simple_strtol(vers, NULL, 0);
> +				num = simple_strtol(vers, &minorp, 0);
> +			if (*minorp == '.') {
> +				if (num < 4)
> +					return -EINVAL;
> +				minor = simple_strtoul(minorp+1, NULL, 0);
> +				if (minor == 0)
> +					return -EINVAL;
> +				if (nfsd_minorversion(minor, sign == '-' ?
> +						     NFSD_CLEAR : NFSD_SET) < 0)
> +					return -EINVAL;
> +				goto next;
> +			}
>  			switch(num) {
>  			case 2:
>  			case 3:
> @@ -826,6 +838,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>  			default:
>  				return -EINVAL;
>  			}
> +		next:
>  			vers += len + 1;
>  			tlen += len;
>  		} while ((len = qword_get(&mesg, vers, size)) > 0);
> @@ -844,6 +857,13 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>  				       num);
>  			sep = " ";
>  		}
> +	if (nfsd_vers(4, NFSD_AVAIL))
> +		for (minor = 1; minor <= NFSD_SUPPORTED_MINOR_VERSION; minor++)
> +			len += sprintf(buf+len, " %c4.%u",
> +					(nfsd_vers(4, NFSD_TEST) &&
> +					 nfsd_minorversion(minor, NFSD_TEST)) ?
> +						'+' : '-',
> +					minor);
>  	len += sprintf(buf+len, "\n");
>  	return len;
>  }
> diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
> index 6ec29c7..5f4cecb 100644
> --- a/fs/nfsd/nfssvc.c
> +++ b/fs/nfsd/nfssvc.c
> @@ -121,6 +121,8 @@ struct svc_program		nfsd_program = {
>  
>  };
>  
> +u32 nfsd_supported_minorversion = NFSD_SUPPORTED_MINOR_VERSION;
> +
>  int nfsd_vers(int vers, enum vers_op change)
>  {
>  	if (vers < NFSD_MINVERS || vers >= NFSD_NRVERS)
> @@ -147,6 +149,28 @@ int nfsd_vers(int vers, enum vers_op change)
>  	}
>  	return 0;
>  }
> +
> +int nfsd_minorversion(u32 minorversion, enum vers_op change)
> +{
> +	if (minorversion > NFSD_SUPPORTED_MINOR_VERSION)
> +		return -1;
> +	switch(change) {
> +	case NFSD_SET:
> +		nfsd_supported_minorversion = minorversion;
> +		break;
> +	case NFSD_CLEAR:
> +		if (minorversion == 0)
> +			return -1;
> +		nfsd_supported_minorversion = minorversion - 1;
> +		break;
> +	case NFSD_TEST:
> +		return minorversion <= nfsd_supported_minorversion;
> +	case NFSD_AVAIL:
> +		return minorversion <= NFSD_SUPPORTED_MINOR_VERSION;
> +	}
> +	return 0;
> +}
> +
>  /*
>   * Maximum number of nfsd processes
>   */
> diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
> index b9e6682..27b9cf5 100644
> --- a/include/linux/nfsd/nfsd.h
> +++ b/include/linux/nfsd/nfsd.h
> @@ -57,6 +57,7 @@ typedef int (*nfsd_dirop_t)(struct inode *, struct dentry *, int, int);
>  extern struct svc_program	nfsd_program;
>  extern struct svc_version	nfsd_version2, nfsd_version3,
>  				nfsd_version4;
> +extern u32			nfsd_supported_minorversion;
>  extern struct mutex		nfsd_mutex;
>  extern struct svc_serv		*nfsd_serv;
>  
> @@ -153,6 +154,7 @@ int nfsd_set_posix_acl(struct svc_fh *, int, struct posix_acl *);
>  
>  enum vers_op {NFSD_SET, NFSD_CLEAR, NFSD_TEST, NFSD_AVAIL };
>  int nfsd_vers(int vers, enum vers_op change);
> +int nfsd_minorversion(u32 minorversion, enum vers_op change);
>  void nfsd_reset_versions(void);
>  int nfsd_create_serv(void);
>  
> _______________________________________________
> pNFS mailing list
> pNFS@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> >> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
> >>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> >>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> >>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> >>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> >>>>> will want to build a kernel with v4.0 but not v4.1?
> 
> Bruce,  with the patch below in place, would it be reasonable to
> remove CONFIG_NFSD_V4_1?

It would be fine with me, but perhaps queuing that up as a separate
patch for 2.6.31 would be better than doing it at the last moment.

(Did you figure out what the problem was?)

--b.

> 
> Benny
> 
> >>>> That's a good question.  I'd love to get rid of it
> >>>> and it seems like like distros, at least RH are going to have it
> >>>> configured-in anyway.
> >>>>
> >>>> If the main reason to turn 4.1 support off is bugs affecting 4.0
> >>>> then I'd much rather fix these bugs rather than hide them.
> >>>>
> >>>>> (And: do we have an interface that allows turning off 4.1 at run-time?
> >>>>> That's more important than the config option.)
> >>>> No, it's still on our todo list.  We haven't thought this completely
> >>>> through, though.  Where would be the best place to implement that?
> >>>> Should this be an export option or an nfsd tunable?
> >>> Imitating (or extending, if possible) nfsd/versions would be one way.
> >>> See fs/nfsd/nfsctl.c:write_versions.  Cc'ing Steved, as I think that was
> >>> originally his work.
> >>>
> >>> --b.
> >>
> >> Sounds great.
> >> Here's a crude untested patch.  Does that what you mean?
> >>
> > 
> > The following works better.
> > 
> > But I still see a problem with it.
> > This all works nicely when the nfs service is stopped (on Fedora 9)
> > but when I restart it all version return to their defaults.
> > I'm probably doing something wrong...
> > 
> > Benny
> > 
> > git diff --stat -p
> >  fs/nfsd/nfs4proc.c        |    2 +-
> >  fs/nfsd/nfsctl.c          |   26 +++++++++++++++++++++++---
> >  fs/nfsd/nfssvc.c          |   24 ++++++++++++++++++++++++
> >  include/linux/nfsd/nfsd.h |    2 ++
> >  4 files changed, 50 insertions(+), 4 deletions(-)
> > 
> > diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> > index a393d38..7619970 100644
> > --- a/fs/nfsd/nfs4proc.c
> > +++ b/fs/nfsd/nfs4proc.c
> > @@ -943,7 +943,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
> >  	 * According to RFC3010, this takes precedence over all other errors.
> >  	 */
> >  	status = nfserr_minor_vers_mismatch;
> > -	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
> > +	if (args->minorversion > nfsd_supported_minorversion)
> >  		goto out;
> >  
> >  	if (!nfs41_op_ordering_ok(args)) {
> > diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
> > index 4d63010..af16849 100644
> > --- a/fs/nfsd/nfsctl.c
> > +++ b/fs/nfsd/nfsctl.c
> > @@ -792,8 +792,9 @@ out_free:
> >  static ssize_t __write_versions(struct file *file, char *buf, size_t size)
> >  {
> >  	char *mesg = buf;
> > -	char *vers, sign;
> > +	char *vers, *minorp, sign;
> >  	int len, num;
> > +	unsigned minor;
> >  	ssize_t tlen = 0;
> >  	char *sep;
> >  
> > @@ -814,9 +815,20 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
> >  		do {
> >  			sign = *vers;
> >  			if (sign == '+' || sign == '-')
> > -				num = simple_strtol((vers+1), NULL, 0);
> > +				num = simple_strtol((vers+1), &minorp, 0);
> >  			else
> > -				num = simple_strtol(vers, NULL, 0);
> > +				num = simple_strtol(vers, &minorp, 0);
> > +			if (*minorp == '.') {
> > +				if (num < 4)
> > +					return -EINVAL;
> > +				minor = simple_strtoul(minorp+1, NULL, 0);
> > +				if (minor == 0)
> > +					return -EINVAL;
> > +				if (nfsd_minorversion(minor, sign == '-' ?
> > +						     NFSD_CLEAR : NFSD_SET) < 0)
> > +					return -EINVAL;
> > +				goto next;
> > +			}
> >  			switch(num) {
> >  			case 2:
> >  			case 3:
> > @@ -826,6 +838,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
> >  			default:
> >  				return -EINVAL;
> >  			}
> > +		next:
> >  			vers += len + 1;
> >  			tlen += len;
> >  		} while ((len = qword_get(&mesg, vers, size)) > 0);
> > @@ -844,6 +857,13 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
> >  				       num);
> >  			sep = " ";
> >  		}
> > +	if (nfsd_vers(4, NFSD_AVAIL))
> > +		for (minor = 1; minor <= NFSD_SUPPORTED_MINOR_VERSION; minor++)
> > +			len += sprintf(buf+len, " %c4.%u",
> > +					(nfsd_vers(4, NFSD_TEST) &&
> > +					 nfsd_minorversion(minor, NFSD_TEST)) ?
> > +						'+' : '-',
> > +					minor);
> >  	len += sprintf(buf+len, "\n");
> >  	return len;
> >  }
> > diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
> > index 6ec29c7..5f4cecb 100644
> > --- a/fs/nfsd/nfssvc.c
> > +++ b/fs/nfsd/nfssvc.c
> > @@ -121,6 +121,8 @@ struct svc_program		nfsd_program = {
> >  
> >  };
> >  
> > +u32 nfsd_supported_minorversion = NFSD_SUPPORTED_MINOR_VERSION;
> > +
> >  int nfsd_vers(int vers, enum vers_op change)
> >  {
> >  	if (vers < NFSD_MINVERS || vers >= NFSD_NRVERS)
> > @@ -147,6 +149,28 @@ int nfsd_vers(int vers, enum vers_op change)
> >  	}
> >  	return 0;
> >  }
> > +
> > +int nfsd_minorversion(u32 minorversion, enum vers_op change)
> > +{
> > +	if (minorversion > NFSD_SUPPORTED_MINOR_VERSION)
> > +		return -1;
> > +	switch(change) {
> > +	case NFSD_SET:
> > +		nfsd_supported_minorversion = minorversion;
> > +		break;
> > +	case NFSD_CLEAR:
> > +		if (minorversion == 0)
> > +			return -1;
> > +		nfsd_supported_minorversion = minorversion - 1;
> > +		break;
> > +	case NFSD_TEST:
> > +		return minorversion <= nfsd_supported_minorversion;
> > +	case NFSD_AVAIL:
> > +		return minorversion <= NFSD_SUPPORTED_MINOR_VERSION;
> > +	}
> > +	return 0;
> > +}
> > +
> >  /*
> >   * Maximum number of nfsd processes
> >   */
> > diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
> > index b9e6682..27b9cf5 100644
> > --- a/include/linux/nfsd/nfsd.h
> > +++ b/include/linux/nfsd/nfsd.h
> > @@ -57,6 +57,7 @@ typedef int (*nfsd_dirop_t)(struct inode *, struct dentry *, int, int);
> >  extern struct svc_program	nfsd_program;
> >  extern struct svc_version	nfsd_version2, nfsd_version3,
> >  				nfsd_version4;
> > +extern u32			nfsd_supported_minorversion;
> >  extern struct mutex		nfsd_mutex;
> >  extern struct svc_serv		*nfsd_serv;
> >  
> > @@ -153,6 +154,7 @@ int nfsd_set_posix_acl(struct svc_fh *, int, struct posix_acl *);
> >  
> >  enum vers_op {NFSD_SET, NFSD_CLEAR, NFSD_TEST, NFSD_AVAIL };
> >  int nfsd_vers(int vers, enum vers_op change);
> > +int nfsd_minorversion(u32 minorversion, enum vers_op change);
> >  void nfsd_reset_versions(void);
> >  int nfsd_create_serv(void);
> >  
> > _______________________________________________
> > pNFS mailing list
> > pNFS@linux-nfs.org
> > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 02, 2009, 16:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
>> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>>> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>>>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
>>>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
>>>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
>>>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
>>>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
>>>>>>> will want to build a kernel with v4.0 but not v4.1?
>> Bruce,  with the patch below in place, would it be reasonable to
>> remove CONFIG_NFSD_V4_1?
> 
> It would be fine with me, but perhaps queuing that up as a separate
> patch for 2.6.31 would be better than doing it at the last moment.

It's not too hard to get rid of it now.
I think it might be better than introducing a new config item
to be removed in the next version.

Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
On the client side minorversion 1 will be used only if the user
explicitly asked for it with mount -o minorversion=1.

Benny

> 
> (Did you figure out what the problem was?)
> 
> --b.
> 
>> Benny
>>
>>>>>> That's a good question.  I'd love to get rid of it
>>>>>> and it seems like like distros, at least RH are going to have it
>>>>>> configured-in anyway.
>>>>>>
>>>>>> If the main reason to turn 4.1 support off is bugs affecting 4.0
>>>>>> then I'd much rather fix these bugs rather than hide them.
>>>>>>
>>>>>>> (And: do we have an interface that allows turning off 4.1 at run-time?
>>>>>>> That's more important than the config option.)
>>>>>> No, it's still on our todo list.  We haven't thought this completely
>>>>>> through, though.  Where would be the best place to implement that?
>>>>>> Should this be an export option or an nfsd tunable?
>>>>> Imitating (or extending, if possible) nfsd/versions would be one way.
>>>>> See fs/nfsd/nfsctl.c:write_versions.  Cc'ing Steved, as I think that was
>>>>> originally his work.
>>>>>
>>>>> --b.
>>>> Sounds great.
>>>> Here's a crude untested patch.  Does that what you mean?
>>>>
>>> The following works better.
>>>
>>> But I still see a problem with it.
>>> This all works nicely when the nfs service is stopped (on Fedora 9)
>>> but when I restart it all version return to their defaults.
>>> I'm probably doing something wrong...
>>>
>>> Benny
>>>
>>> git diff --stat -p
>>>  fs/nfsd/nfs4proc.c        |    2 +-
>>>  fs/nfsd/nfsctl.c          |   26 +++++++++++++++++++++++---
>>>  fs/nfsd/nfssvc.c          |   24 ++++++++++++++++++++++++
>>>  include/linux/nfsd/nfsd.h |    2 ++
>>>  4 files changed, 50 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>>> index a393d38..7619970 100644
>>> --- a/fs/nfsd/nfs4proc.c
>>> +++ b/fs/nfsd/nfs4proc.c
>>> @@ -943,7 +943,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>>>  	 * According to RFC3010, this takes precedence over all other errors.
>>>  	 */
>>>  	status = nfserr_minor_vers_mismatch;
>>> -	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
>>> +	if (args->minorversion > nfsd_supported_minorversion)
>>>  		goto out;
>>>  
>>>  	if (!nfs41_op_ordering_ok(args)) {
>>> diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
>>> index 4d63010..af16849 100644
>>> --- a/fs/nfsd/nfsctl.c
>>> +++ b/fs/nfsd/nfsctl.c
>>> @@ -792,8 +792,9 @@ out_free:
>>>  static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>>>  {
>>>  	char *mesg = buf;
>>> -	char *vers, sign;
>>> +	char *vers, *minorp, sign;
>>>  	int len, num;
>>> +	unsigned minor;
>>>  	ssize_t tlen = 0;
>>>  	char *sep;
>>>  
>>> @@ -814,9 +815,20 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>>>  		do {
>>>  			sign = *vers;
>>>  			if (sign == '+' || sign == '-')
>>> -				num = simple_strtol((vers+1), NULL, 0);
>>> +				num = simple_strtol((vers+1), &minorp, 0);
>>>  			else
>>> -				num = simple_strtol(vers, NULL, 0);
>>> +				num = simple_strtol(vers, &minorp, 0);
>>> +			if (*minorp == '.') {
>>> +				if (num < 4)
>>> +					return -EINVAL;
>>> +				minor = simple_strtoul(minorp+1, NULL, 0);
>>> +				if (minor == 0)
>>> +					return -EINVAL;
>>> +				if (nfsd_minorversion(minor, sign == '-' ?
>>> +						     NFSD_CLEAR : NFSD_SET) < 0)
>>> +					return -EINVAL;
>>> +				goto next;
>>> +			}
>>>  			switch(num) {
>>>  			case 2:
>>>  			case 3:
>>> @@ -826,6 +838,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>>>  			default:
>>>  				return -EINVAL;
>>>  			}
>>> +		next:
>>>  			vers += len + 1;
>>>  			tlen += len;
>>>  		} while ((len = qword_get(&mesg, vers, size)) > 0);
>>> @@ -844,6 +857,13 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
>>>  				       num);
>>>  			sep = " ";
>>>  		}
>>> +	if (nfsd_vers(4, NFSD_AVAIL))
>>> +		for (minor = 1; minor <= NFSD_SUPPORTED_MINOR_VERSION; minor++)
>>> +			len += sprintf(buf+len, " %c4.%u",
>>> +					(nfsd_vers(4, NFSD_TEST) &&
>>> +					 nfsd_minorversion(minor, NFSD_TEST)) ?
>>> +						'+' : '-',
>>> +					minor);
>>>  	len += sprintf(buf+len, "\n");
>>>  	return len;
>>>  }
>>> diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
>>> index 6ec29c7..5f4cecb 100644
>>> --- a/fs/nfsd/nfssvc.c
>>> +++ b/fs/nfsd/nfssvc.c
>>> @@ -121,6 +121,8 @@ struct svc_program		nfsd_program = {
>>>  
>>>  };
>>>  
>>> +u32 nfsd_supported_minorversion = NFSD_SUPPORTED_MINOR_VERSION;
>>> +
>>>  int nfsd_vers(int vers, enum vers_op change)
>>>  {
>>>  	if (vers < NFSD_MINVERS || vers >= NFSD_NRVERS)
>>> @@ -147,6 +149,28 @@ int nfsd_vers(int vers, enum vers_op change)
>>>  	}
>>>  	return 0;
>>>  }
>>> +
>>> +int nfsd_minorversion(u32 minorversion, enum vers_op change)
>>> +{
>>> +	if (minorversion > NFSD_SUPPORTED_MINOR_VERSION)
>>> +		return -1;
>>> +	switch(change) {
>>> +	case NFSD_SET:
>>> +		nfsd_supported_minorversion = minorversion;
>>> +		break;
>>> +	case NFSD_CLEAR:
>>> +		if (minorversion == 0)
>>> +			return -1;
>>> +		nfsd_supported_minorversion = minorversion - 1;
>>> +		break;
>>> +	case NFSD_TEST:
>>> +		return minorversion <= nfsd_supported_minorversion;
>>> +	case NFSD_AVAIL:
>>> +		return minorversion <= NFSD_SUPPORTED_MINOR_VERSION;
>>> +	}
>>> +	return 0;
>>> +}
>>> +
>>>  /*
>>>   * Maximum number of nfsd processes
>>>   */
>>> diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
>>> index b9e6682..27b9cf5 100644
>>> --- a/include/linux/nfsd/nfsd.h
>>> +++ b/include/linux/nfsd/nfsd.h
>>> @@ -57,6 +57,7 @@ typedef int (*nfsd_dirop_t)(struct inode *, struct dentry *, int, int);
>>>  extern struct svc_program	nfsd_program;
>>>  extern struct svc_version	nfsd_version2, nfsd_version3,
>>>  				nfsd_version4;
>>> +extern u32			nfsd_supported_minorversion;
>>>  extern struct mutex		nfsd_mutex;
>>>  extern struct svc_serv		*nfsd_serv;
>>>  
>>> @@ -153,6 +154,7 @@ int nfsd_set_posix_acl(struct svc_fh *, int, struct posix_acl *);
>>>  
>>>  enum vers_op {NFSD_SET, NFSD_CLEAR, NFSD_TEST, NFSD_AVAIL };
>>>  int nfsd_vers(int vers, enum vers_op change);
>>> +int nfsd_minorversion(u32 minorversion, enum vers_op change);
>>>  void nfsd_reset_versions(void);
>>>  int nfsd_create_serv(void);
>>>  
>>> _______________________________________________
>>> pNFS mailing list
>>> pNFS@linux-nfs.org
>>> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Trond Myklebust]

On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 16:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
> >> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> >>> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> >>>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
> >>>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> >>>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> >>>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> >>>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> >>>>>>> will want to build a kernel with v4.0 but not v4.1?
> >> Bruce,  with the patch below in place, would it be reasonable to
> >> remove CONFIG_NFSD_V4_1?
> > 
> > It would be fine with me, but perhaps queuing that up as a separate
> > patch for 2.6.31 would be better than doing it at the last moment.
> 
> It's not too hard to get rid of it now.
> I think it might be better than introducing a new config item
> to be removed in the next version.
> 
> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> On the client side minorversion 1 will be used only if the user
> explicitly asked for it with mount -o minorversion=1.

I'd feel more comfortable with being able to compile it out until the
stability of the code has been established. I'd certainly want to be
able to do that on the server side, since it has no other means to
restrict the protocol version should it turn out that NFSv4.1 has some
fatal condition.

Cheers
   Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> > On Apr. 02, 2009, 16:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
> > >> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > >>> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > >>>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > >>>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
> > >>>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > >>>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> > >>>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> > >>>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> > >>>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> > >>>>>>> will want to build a kernel with v4.0 but not v4.1?
> > >> Bruce,  with the patch below in place, would it be reasonable to
> > >> remove CONFIG_NFSD_V4_1?
> > > 
> > > It would be fine with me, but perhaps queuing that up as a separate
> > > patch for 2.6.31 would be better than doing it at the last moment.
> > 
> > It's not too hard to get rid of it now.
> > I think it might be better than introducing a new config item
> > to be removed in the next version.
> > 
> > Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> > On the client side minorversion 1 will be used only if the user
> > explicitly asked for it with mount -o minorversion=1.
> 
> I'd feel more comfortable with being able to compile it out until the
> stability of the code has been established. I'd certainly want to be
> able to do that on the server side, since it has no other means to
> restrict the protocol version should it turn out that NFSv4.1 has some
> fatal condition.

I think it's acceptable given an interface that allows choosing the
supported minorversion at runtime (and that defaults 4.1 to off).

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Trond Myklebust]

On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> > On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> > > On Apr. 02, 2009, 16:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > > On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
> > > >> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > > >>> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > > >>>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > >>>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
> > > >>>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > >>>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> > > >>>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> > > >>>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> > > >>>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> > > >>>>>>> will want to build a kernel with v4.0 but not v4.1?
> > > >> Bruce,  with the patch below in place, would it be reasonable to
> > > >> remove CONFIG_NFSD_V4_1?
> > > > 
> > > > It would be fine with me, but perhaps queuing that up as a separate
> > > > patch for 2.6.31 would be better than doing it at the last moment.
> > > 
> > > It's not too hard to get rid of it now.
> > > I think it might be better than introducing a new config item
> > > to be removed in the next version.
> > > 
> > > Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> > > On the client side minorversion 1 will be used only if the user
> > > explicitly asked for it with mount -o minorversion=1.
> > 
> > I'd feel more comfortable with being able to compile it out until the
> > stability of the code has been established. I'd certainly want to be
> > able to do that on the server side, since it has no other means to
> > restrict the protocol version should it turn out that NFSv4.1 has some
> > fatal condition.
> 
> I think it's acceptable given an interface that allows choosing the
> supported minorversion at runtime (and that defaults 4.1 to off).

Is there such an interface on the server?

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
> > On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> > > On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> > > > On Apr. 02, 2009, 16:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > > > On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
> > > > >> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > > > >>> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > > > >>>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > > >>>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
> > > > >>>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > > >>>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
> > > > >>>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
> > > > >>>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
> > > > >>>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
> > > > >>>>>>> will want to build a kernel with v4.0 but not v4.1?
> > > > >> Bruce,  with the patch below in place, would it be reasonable to
> > > > >> remove CONFIG_NFSD_V4_1?
> > > > > 
> > > > > It would be fine with me, but perhaps queuing that up as a separate
> > > > > patch for 2.6.31 would be better than doing it at the last moment.
> > > > 
> > > > It's not too hard to get rid of it now.
> > > > I think it might be better than introducing a new config item
> > > > to be removed in the next version.
> > > > 
> > > > Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> > > > On the client side minorversion 1 will be used only if the user
> > > > explicitly asked for it with mount -o minorversion=1.
> > > 
> > > I'd feel more comfortable with being able to compile it out until the
> > > stability of the code has been established. I'd certainly want to be
> > > able to do that on the server side, since it has no other means to
> > > restrict the protocol version should it turn out that NFSv4.1 has some
> > > fatal condition.
> > 
> > I think it's acceptable given an interface that allows choosing the
> > supported minorversion at runtime (and that defaults 4.1 to off).
> 
> Is there such an interface on the server?

That's the patch Benny just posted.  It seems like a pretty simple
extension of the existing version-choosing interface
(/proc/fs/nfsd/versions), though I think the version he posted defaults
4.1 to on?  I need to take another look.

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
>>>>> On Apr. 02, 2009, 16:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>> On Thu, Apr 02, 2009 at 12:18:46PM +0300, Benny Halevy wrote:
>>>>>>> On Apr. 01, 2009, 18:32 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>>>>>>>> On Apr. 01, 2009, 17:07 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>>>>>>>>> On Apr. 01, 2009, 16:10 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>>>>> On Wed, Apr 01, 2009 at 11:31:21AM +0300, Benny Halevy wrote:
>>>>>>>>>>> On Apr. 01, 2009, 7:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>>>>>>> On Sat, Mar 28, 2009 at 11:31:09AM +0300, Benny Halevy wrote:
>>>>>>>>>>>>> Added CONFIG_NFSD_V4_1 and made it depend upon NFSD_V4 and EXPERIMENTAL
>>>>>>>>>>>>> Indicate that CONFIG_NFS_V4_1 is for NFS developers at the moment
>>>>>>>>>>>> Stupid question: do we need CONFIG_NFSD_V4_1 at all?  How many people
>>>>>>>>>>>> will want to build a kernel with v4.0 but not v4.1?
>>>>>>> Bruce,  with the patch below in place, would it be reasonable to
>>>>>>> remove CONFIG_NFSD_V4_1?
>>>>>> It would be fine with me, but perhaps queuing that up as a separate
>>>>>> patch for 2.6.31 would be better than doing it at the last moment.
>>>>> It's not too hard to get rid of it now.
>>>>> I think it might be better than introducing a new config item
>>>>> to be removed in the next version.
>>>>>
>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
>>>>> On the client side minorversion 1 will be used only if the user
>>>>> explicitly asked for it with mount -o minorversion=1.
>>>> I'd feel more comfortable with being able to compile it out until the
>>>> stability of the code has been established. I'd certainly want to be
>>>> able to do that on the server side, since it has no other means to
>>>> restrict the protocol version should it turn out that NFSv4.1 has some
>>>> fatal condition.
>>> I think it's acceptable given an interface that allows choosing the
>>> supported minorversion at runtime (and that defaults 4.1 to off).
>> Is there such an interface on the server?
> 
> That's the patch Benny just posted.  It seems like a pretty simple
> extension of the existing version-choosing interface
> (/proc/fs/nfsd/versions), though I think the version he posted defaults
> 4.1 to on?  I need to take another look.

That's right.  This can be changed trivially,
but I'll have to find and work the reason why the settings reset on
service nfs restart, otherwise it will reset itself.

Benny

> 
> --b.
> _______________________________________________
> pNFS mailing list
> pNFS@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
> >> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
> >>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> >>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> >>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> >>>>> On the client side minorversion 1 will be used only if the user
> >>>>> explicitly asked for it with mount -o minorversion=1.
> >>>> I'd feel more comfortable with being able to compile it out until the
> >>>> stability of the code has been established. I'd certainly want to be
> >>>> able to do that on the server side, since it has no other means to
> >>>> restrict the protocol version should it turn out that NFSv4.1 has some
> >>>> fatal condition.
> >>> I think it's acceptable given an interface that allows choosing the
> >>> supported minorversion at runtime (and that defaults 4.1 to off).
> >> Is there such an interface on the server?
> > 
> > That's the patch Benny just posted.  It seems like a pretty simple
> > extension of the existing version-choosing interface
> > (/proc/fs/nfsd/versions), though I think the version he posted defaults
> > 4.1 to on?  I need to take another look.
> 
> That's right.  This can be changed trivially,
> but I'll have to find and work the reason why the settings reset on
> service nfs restart, otherwise it will reset itself.

Maybe you're starting and stopping the server using distro init scripts
that load and unload the nfsd module?

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
>>>>>>> On the client side minorversion 1 will be used only if the user
>>>>>>> explicitly asked for it with mount -o minorversion=1.
>>>>>> I'd feel more comfortable with being able to compile it out until the
>>>>>> stability of the code has been established. I'd certainly want to be
>>>>>> able to do that on the server side, since it has no other means to
>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
>>>>>> fatal condition.
>>>>> I think it's acceptable given an interface that allows choosing the
>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
>>>> Is there such an interface on the server?
>>> That's the patch Benny just posted.  It seems like a pretty simple
>>> extension of the existing version-choosing interface
>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
>>> 4.1 to on?  I need to take another look.
>> That's right.  This can be changed trivially,
>> but I'll have to find and work the reason why the settings reset on
>> service nfs restart, otherwise it will reset itself.
> 
> Maybe you're starting and stopping the server using distro init scripts
> that load and unload the nfsd module?

I'm using redhat's /etc/init.d/nfs
My impression was that it doesn't unload the nfsd module when the service
is stopped.  However it does run modprobe on "start".  I wonder if that's
causing the module to reload.

Here's their script for Fedora 9:

#!/bin/sh
#
# nfs           This shell script takes care of starting and stopping
#               the NFS services.
#
# chkconfig: - 60 20
# description: NFS is a popular protocol for file sharing across TCP/IP \
#              networks. This service provides NFS server functionality, \
#              which is configured via the /etc/exports file.
# probe: true
# config: /etc/sysconfig/nfs

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
if [ ! -f /etc/sysconfig/network ]; then
    exit 6
fi

. /etc/sysconfig/network

# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 6

[ -x /usr/sbin/rpc.nfsd ] || exit 5
[ -x /usr/sbin/rpc.mountd ] || exit 5
[ -x /usr/sbin/exportfs ] || exit 5

# Don't fail if /etc/exports doesn't exist; create a bare-bones version and continue.
[ -r /etc/exports ] || \
    { touch /etc/exports && chmod u+rw,g+r,o+r /etc/exports ; } || \
    { echo "/etc/exports does not exist" ; exit 0 ; }

# Check for and source configuration file otherwise set defaults
[ -f /etc/sysconfig/nfs ] && . /etc/sysconfig/nfs

[ -z "$MOUNTD_NFS_V2" ] && MOUNTD_NFS_V2=default
[ -z "$MOUNTD_NFS_V3" ] && MOUNTD_NFS_V3=default

# Number of servers to be started by default
[ -z "$RPCNFSDCOUNT" ] && RPCNFSDCOUNT=8

# Remote quota server
[ -z "$RQUOTAD" ] && RQUOTAD=`type -path rpc.rquotad`

RETVAL=0

# See how we were called.
case "$1" in
  start)
	# Start daemons.
	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd start

	# Load the nfsd module so /proc/fs/nfsd will exist
	[ "$NFSD_MODULE" != "noload" ] && {
		[ -x /sbin/modprobe ] && /sbin/modprobe nfsd
	}

	action $"Starting NFS services: " /usr/sbin/exportfs -r

	# Set the ports lockd should listen on
	if [ -n "$LOCKD_TCPPORT" ]; then
	    /sbin/sysctl -w fs.nfs.nlm_tcpport=$LOCKD_TCPPORT >/dev/null 2>&1
	fi
	if [ -n "$LOCKD_UDPPORT" ]; then
	    /sbin/sysctl -w fs.nfs.nlm_udpport=$LOCKD_UDPPORT >/dev/null 2>&1
	fi

	if [ -n "$RQUOTAD" -a "$RQUOTAD" != "no" ]; then
	    echo -n $"Starting NFS quotas: "
		[ -n "$RQUOTAD_PORT" ] \
			&& RPCRQUOTADOPTS="$RPCRQUOTADOPTS -p $RQUOTAD_PORT"
	    daemon rpc.rquotad $RPCRQUOTADOPTS
		RETVAL=$?
	    echo
	fi
	echo -n $"Starting NFS daemon: "
	daemon rpc.nfsd $RPCNFSDARGS $RPCNFSDCOUNT
	RETVAL=$?
	echo
	[ $RETVAL -ne 0 ] && exit $RETVAL

	[ -n "$MOUNTD_PORT" ] \
	&& RPCMOUNTDOPTS="$RPCMOUNTDOPTS -p $MOUNTD_PORT"

	case $MOUNTD_NFS_V1 in
	no|NO)
	    RPCMOUNTDOPTS="$RPCMOUNTDOPTS --no-nfs-version 1" ;;
	esac

	case $MOUNTD_NFS_V2 in
	no|NO)
	    RPCMOUNTDOPTS="$RPCMOUNTDOPTS --no-nfs-version 2" ;;
	esac

	case $MOUNTD_NFS_V3 in
	no|NO)
	    RPCMOUNTDOPTS="$RPCMOUNTDOPTS --no-nfs-version 3" ;;
	esac

	echo -n $"Starting NFS mountd: "
	daemon rpc.mountd $RPCMOUNTDOPTS
	RETVAL=$?
	echo
	touch /var/lock/subsys/nfs

	# Let rpc.idmapd know that rpc.mountd just started
	[ -x /usr/sbin/rpc.idmapd ] && /sbin/service rpcidmapd condstart

	;;
  stop)
	# Stop daemons.
	echo -n $"Shutting down NFS mountd: "
	killproc rpc.mountd
	echo
	echo -n $"Shutting down NFS daemon: "
	killproc nfsd -2
	echo
	if [ -n "$RQUOTAD" -a "$RQUOTAD" != "no" ]; then
		echo -n $"Shutting down NFS quotas: "
		killproc rpc.rquotad
		RETVAL=$?
		echo
	fi
	# Reset the lockd ports if they were set
	if [ -n "$LOCKD_TCPPORT" ]; then
	    /sbin/sysctl -w fs.nfs.nlm_tcpport=0 >/dev/null 2>&1
	fi
	if [ -n "$LOCKD_UDPPORT" ]; then
	    /sbin/sysctl -w fs.nfs.nlm_udpport=0 >/dev/null 2>&1
	fi
	# Do it the last so that clients can still access the server
	# when the server is running.
	cnt=`/usr/sbin/exportfs -v | /usr/bin/wc -l`	
	if [ $cnt -gt 0 ]; then 
		action $"Shutting down NFS services: " /usr/sbin/exportfs -au
	else
		action $"Shutting down NFS services: " /bin/false
	fi
	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd stop
	rm -f /var/lock/subsys/nfs
	;;
  status)
	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd status
	status rpc.mountd
	status nfsd
	if [ -n "$RQUOTAD" -a "$RQUOTAD" != "no" ]; then
		status rpc.rquotad
	fi
	RETVAL=$?
	;;
  restart)
	$0 stop
	$0 start
	;;
  reload)
	/usr/sbin/exportfs -r
	touch /var/lock/subsys/nfs
	;;
  probe)
	if [ ! -f /var/lock/subsys/nfs ] ; then
	  echo $"start"; exit 0
	fi
	/sbin/pidof rpc.mountd >/dev/null 2>&1; MOUNTD="$?"
	/sbin/pidof nfsd >/dev/null 2>&1; NFSD="$?"
	if [ $MOUNTD = 1 -o $NFSD = 1 ] ; then
	  echo $"restart"; exit 0
	fi
	if [ /etc/exports -nt /var/lock/subsys/nfs ] ; then
	  echo $"reload"; exit 0
	fi
	;;
  condrestart)
	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd condrestart
  	[ -f /var/lock/subsys/nfs ] && {
		$0 stop
		$0 start
	}
	;;
  *)
	echo $"Usage: nfs {start|stop|status|restart|reload|condrestart}"
	RETVAL=3
	;;
esac

exit $RETVAL



> 
> --b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 02, 2009, 20:45 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
>>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
>>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
>>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
>>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
>>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
>>>>>>>> On the client side minorversion 1 will be used only if the user
>>>>>>>> explicitly asked for it with mount -o minorversion=1.
>>>>>>> I'd feel more comfortable with being able to compile it out until the
>>>>>>> stability of the code has been established. I'd certainly want to be
>>>>>>> able to do that on the server side, since it has no other means to
>>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
>>>>>>> fatal condition.
>>>>>> I think it's acceptable given an interface that allows choosing the
>>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
>>>>> Is there such an interface on the server?
>>>> That's the patch Benny just posted.  It seems like a pretty simple
>>>> extension of the existing version-choosing interface
>>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
>>>> 4.1 to on?  I need to take another look.
>>> That's right.  This can be changed trivially,
>>> but I'll have to find and work the reason why the settings reset on
>>> service nfs restart, otherwise it will reset itself.
>> Maybe you're starting and stopping the server using distro init scripts
>> that load and unload the nfsd module?
> 
> I'm using redhat's /etc/init.d/nfs
> My impression was that it doesn't unload the nfsd module when the service
> is stopped.  However it does run modprobe on "start".  I wonder if that's
> causing the module to reload.

Hmm, after "stop" nfsd is up and must stay up to keep serving
/proc/fs/nfsd/versions.

Giving modprobe the --first-time option reports:
FATAL: Module nfsd already in kernel.

Benny

> 
> Here's their script for Fedora 9:
> 
> #!/bin/sh
> #
> # nfs           This shell script takes care of starting and stopping
> #               the NFS services.
> #
> # chkconfig: - 60 20
> # description: NFS is a popular protocol for file sharing across TCP/IP \
> #              networks. This service provides NFS server functionality, \
> #              which is configured via the /etc/exports file.
> # probe: true
> # config: /etc/sysconfig/nfs
> 
> # Source function library.
> . /etc/rc.d/init.d/functions
> 
> # Source networking configuration.
> if [ ! -f /etc/sysconfig/network ]; then
>     exit 6
> fi
> 
> . /etc/sysconfig/network
> 
> # Check that networking is up.
> [ "${NETWORKING}" = "no" ] && exit 6
> 
> [ -x /usr/sbin/rpc.nfsd ] || exit 5
> [ -x /usr/sbin/rpc.mountd ] || exit 5
> [ -x /usr/sbin/exportfs ] || exit 5
> 
> # Don't fail if /etc/exports doesn't exist; create a bare-bones version and continue.
> [ -r /etc/exports ] || \
>     { touch /etc/exports && chmod u+rw,g+r,o+r /etc/exports ; } || \
>     { echo "/etc/exports does not exist" ; exit 0 ; }
> 
> # Check for and source configuration file otherwise set defaults
> [ -f /etc/sysconfig/nfs ] && . /etc/sysconfig/nfs
> 
> [ -z "$MOUNTD_NFS_V2" ] && MOUNTD_NFS_V2=default
> [ -z "$MOUNTD_NFS_V3" ] && MOUNTD_NFS_V3=default
> 
> # Number of servers to be started by default
> [ -z "$RPCNFSDCOUNT" ] && RPCNFSDCOUNT=8
> 
> # Remote quota server
> [ -z "$RQUOTAD" ] && RQUOTAD=`type -path rpc.rquotad`
> 
> RETVAL=0
> 
> # See how we were called.
> case "$1" in
>   start)
> 	# Start daemons.
> 	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd start
> 
> 	# Load the nfsd module so /proc/fs/nfsd will exist
> 	[ "$NFSD_MODULE" != "noload" ] && {
> 		[ -x /sbin/modprobe ] && /sbin/modprobe nfsd
> 	}
> 
> 	action $"Starting NFS services: " /usr/sbin/exportfs -r
> 
> 	# Set the ports lockd should listen on
> 	if [ -n "$LOCKD_TCPPORT" ]; then
> 	    /sbin/sysctl -w fs.nfs.nlm_tcpport=$LOCKD_TCPPORT >/dev/null 2>&1
> 	fi
> 	if [ -n "$LOCKD_UDPPORT" ]; then
> 	    /sbin/sysctl -w fs.nfs.nlm_udpport=$LOCKD_UDPPORT >/dev/null 2>&1
> 	fi
> 
> 	if [ -n "$RQUOTAD" -a "$RQUOTAD" != "no" ]; then
> 	    echo -n $"Starting NFS quotas: "
> 		[ -n "$RQUOTAD_PORT" ] \
> 			&& RPCRQUOTADOPTS="$RPCRQUOTADOPTS -p $RQUOTAD_PORT"
> 	    daemon rpc.rquotad $RPCRQUOTADOPTS
> 		RETVAL=$?
> 	    echo
> 	fi
> 	echo -n $"Starting NFS daemon: "
> 	daemon rpc.nfsd $RPCNFSDARGS $RPCNFSDCOUNT
> 	RETVAL=$?
> 	echo
> 	[ $RETVAL -ne 0 ] && exit $RETVAL
> 
> 	[ -n "$MOUNTD_PORT" ] \
> 	&& RPCMOUNTDOPTS="$RPCMOUNTDOPTS -p $MOUNTD_PORT"
> 
> 	case $MOUNTD_NFS_V1 in
> 	no|NO)
> 	    RPCMOUNTDOPTS="$RPCMOUNTDOPTS --no-nfs-version 1" ;;
> 	esac
> 
> 	case $MOUNTD_NFS_V2 in
> 	no|NO)
> 	    RPCMOUNTDOPTS="$RPCMOUNTDOPTS --no-nfs-version 2" ;;
> 	esac
> 
> 	case $MOUNTD_NFS_V3 in
> 	no|NO)
> 	    RPCMOUNTDOPTS="$RPCMOUNTDOPTS --no-nfs-version 3" ;;
> 	esac
> 
> 	echo -n $"Starting NFS mountd: "
> 	daemon rpc.mountd $RPCMOUNTDOPTS
> 	RETVAL=$?
> 	echo
> 	touch /var/lock/subsys/nfs
> 
> 	# Let rpc.idmapd know that rpc.mountd just started
> 	[ -x /usr/sbin/rpc.idmapd ] && /sbin/service rpcidmapd condstart
> 
> 	;;
>   stop)
> 	# Stop daemons.
> 	echo -n $"Shutting down NFS mountd: "
> 	killproc rpc.mountd
> 	echo
> 	echo -n $"Shutting down NFS daemon: "
> 	killproc nfsd -2
> 	echo
> 	if [ -n "$RQUOTAD" -a "$RQUOTAD" != "no" ]; then
> 		echo -n $"Shutting down NFS quotas: "
> 		killproc rpc.rquotad
> 		RETVAL=$?
> 		echo
> 	fi
> 	# Reset the lockd ports if they were set
> 	if [ -n "$LOCKD_TCPPORT" ]; then
> 	    /sbin/sysctl -w fs.nfs.nlm_tcpport=0 >/dev/null 2>&1
> 	fi
> 	if [ -n "$LOCKD_UDPPORT" ]; then
> 	    /sbin/sysctl -w fs.nfs.nlm_udpport=0 >/dev/null 2>&1
> 	fi
> 	# Do it the last so that clients can still access the server
> 	# when the server is running.
> 	cnt=`/usr/sbin/exportfs -v | /usr/bin/wc -l`	
> 	if [ $cnt -gt 0 ]; then 
> 		action $"Shutting down NFS services: " /usr/sbin/exportfs -au
> 	else
> 		action $"Shutting down NFS services: " /bin/false
> 	fi
> 	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd stop
> 	rm -f /var/lock/subsys/nfs
> 	;;
>   status)
> 	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd status
> 	status rpc.mountd
> 	status nfsd
> 	if [ -n "$RQUOTAD" -a "$RQUOTAD" != "no" ]; then
> 		status rpc.rquotad
> 	fi
> 	RETVAL=$?
> 	;;
>   restart)
> 	$0 stop
> 	$0 start
> 	;;
>   reload)
> 	/usr/sbin/exportfs -r
> 	touch /var/lock/subsys/nfs
> 	;;
>   probe)
> 	if [ ! -f /var/lock/subsys/nfs ] ; then
> 	  echo $"start"; exit 0
> 	fi
> 	/sbin/pidof rpc.mountd >/dev/null 2>&1; MOUNTD="$?"
> 	/sbin/pidof nfsd >/dev/null 2>&1; NFSD="$?"
> 	if [ $MOUNTD = 1 -o $NFSD = 1 ] ; then
> 	  echo $"restart"; exit 0
> 	fi
> 	if [ /etc/exports -nt /var/lock/subsys/nfs ] ; then
> 	  echo $"reload"; exit 0
> 	fi
> 	;;
>   condrestart)
> 	[ -x /usr/sbin/rpc.svcgssd ] && /sbin/service rpcsvcgssd condrestart
>   	[ -f /var/lock/subsys/nfs ] && {
> 		$0 stop
> 		$0 start
> 	}
> 	;;
>   *)
> 	echo $"Usage: nfs {start|stop|status|restart|reload|condrestart}"
> 	RETVAL=3
> 	;;
> esac
> 
> exit $RETVAL
> 
> 
> 
>> --b.
> _______________________________________________
> pNFS mailing list
> pNFS@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs


-- 
Benny Halevy
Software Architect
Panasas, Inc.
bhalevy@panasas.com
Tel/Fax: +972-3-647-8340
Mobile: +972-54-802-8340

Panasas: The Leader in Parallel Storage
www.panasas.com

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 08:52:40PM +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 20:45 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
> >>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
> >>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
> >>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> >>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> >>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> >>>>>>>> On the client side minorversion 1 will be used only if the user
> >>>>>>>> explicitly asked for it with mount -o minorversion=1.
> >>>>>>> I'd feel more comfortable with being able to compile it out until the
> >>>>>>> stability of the code has been established. I'd certainly want to be
> >>>>>>> able to do that on the server side, since it has no other means to
> >>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
> >>>>>>> fatal condition.
> >>>>>> I think it's acceptable given an interface that allows choosing the
> >>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
> >>>>> Is there such an interface on the server?
> >>>> That's the patch Benny just posted.  It seems like a pretty simple
> >>>> extension of the existing version-choosing interface
> >>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
> >>>> 4.1 to on?  I need to take another look.
> >>> That's right.  This can be changed trivially,
> >>> but I'll have to find and work the reason why the settings reset on
> >>> service nfs restart, otherwise it will reset itself.
> >> Maybe you're starting and stopping the server using distro init scripts
> >> that load and unload the nfsd module?
> > 
> > I'm using redhat's /etc/init.d/nfs
> > My impression was that it doesn't unload the nfsd module when the service
> > is stopped.  However it does run modprobe on "start".  I wonder if that's
> > causing the module to reload.
> 
> Hmm, after "stop" nfsd is up and must stay up to keep serving
> /proc/fs/nfsd/versions.
> 
> Giving modprobe the --first-time option reports:
> FATAL: Module nfsd already in kernel.

Yeah, actually, /proc/fs/nfsd/versions is probably getting written
somewhere in there.  That may be the effect of the --no-nfs-version
mountd argument.

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 01:55:17PM -0400, bfields wrote:
> On Thu, Apr 02, 2009 at 08:52:40PM +0300, Benny Halevy wrote:
> > On Apr. 02, 2009, 20:45 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> > > On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > >> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
> > >>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > >>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
> > >>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
> > >>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> > >>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> > >>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> > >>>>>>>> On the client side minorversion 1 will be used only if the user
> > >>>>>>>> explicitly asked for it with mount -o minorversion=1.
> > >>>>>>> I'd feel more comfortable with being able to compile it out until the
> > >>>>>>> stability of the code has been established. I'd certainly want to be
> > >>>>>>> able to do that on the server side, since it has no other means to
> > >>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
> > >>>>>>> fatal condition.
> > >>>>>> I think it's acceptable given an interface that allows choosing the
> > >>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
> > >>>>> Is there such an interface on the server?
> > >>>> That's the patch Benny just posted.  It seems like a pretty simple
> > >>>> extension of the existing version-choosing interface
> > >>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
> > >>>> 4.1 to on?  I need to take another look.
> > >>> That's right.  This can be changed trivially,
> > >>> but I'll have to find and work the reason why the settings reset on
> > >>> service nfs restart, otherwise it will reset itself.
> > >> Maybe you're starting and stopping the server using distro init scripts
> > >> that load and unload the nfsd module?
> > > 
> > > I'm using redhat's /etc/init.d/nfs
> > > My impression was that it doesn't unload the nfsd module when the service
> > > is stopped.  However it does run modprobe on "start".  I wonder if that's
> > > causing the module to reload.
> > 
> > Hmm, after "stop" nfsd is up and must stay up to keep serving
> > /proc/fs/nfsd/versions.
> > 
> > Giving modprobe the --first-time option reports:
> > FATAL: Module nfsd already in kernel.
> 
> Yeah, actually, /proc/fs/nfsd/versions is probably getting written
> somewhere in there.  That may be the effect of the --no-nfs-version
> mountd argument.

Whoops, sorry, rpc.nfsd: it should be the --no-nfs-version argument to
rpc.nfsd.  See nfssvc_versbits() in nfs-utils/support/nfs/nfssvc.c.

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 02, 2009, 20:58 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 01:55:17PM -0400, bfields wrote:
>> On Thu, Apr 02, 2009 at 08:52:40PM +0300, Benny Halevy wrote:
>>> On Apr. 02, 2009, 20:45 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>>>> On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
>>>>>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
>>>>>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
>>>>>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
>>>>>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
>>>>>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
>>>>>>>>>>> On the client side minorversion 1 will be used only if the user
>>>>>>>>>>> explicitly asked for it with mount -o minorversion=1.
>>>>>>>>>> I'd feel more comfortable with being able to compile it out until the
>>>>>>>>>> stability of the code has been established. I'd certainly want to be
>>>>>>>>>> able to do that on the server side, since it has no other means to
>>>>>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
>>>>>>>>>> fatal condition.
>>>>>>>>> I think it's acceptable given an interface that allows choosing the
>>>>>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
>>>>>>>> Is there such an interface on the server?
>>>>>>> That's the patch Benny just posted.  It seems like a pretty simple
>>>>>>> extension of the existing version-choosing interface
>>>>>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
>>>>>>> 4.1 to on?  I need to take another look.
>>>>>> That's right.  This can be changed trivially,
>>>>>> but I'll have to find and work the reason why the settings reset on
>>>>>> service nfs restart, otherwise it will reset itself.
>>>>> Maybe you're starting and stopping the server using distro init scripts
>>>>> that load and unload the nfsd module?
>>>> I'm using redhat's /etc/init.d/nfs
>>>> My impression was that it doesn't unload the nfsd module when the service
>>>> is stopped.  However it does run modprobe on "start".  I wonder if that's
>>>> causing the module to reload.
>>> Hmm, after "stop" nfsd is up and must stay up to keep serving
>>> /proc/fs/nfsd/versions.
>>>
>>> Giving modprobe the --first-time option reports:
>>> FATAL: Module nfsd already in kernel.
>> Yeah, actually, /proc/fs/nfsd/versions is probably getting written
>> somewhere in there.  That may be the effect of the --no-nfs-version
>> mountd argument.
> 
> Whoops, sorry, rpc.nfsd: it should be the --no-nfs-version argument to
> rpc.nfsd.  See nfssvc_versbits() in nfs-utils/support/nfs/nfssvc.c.

OK. I see that now.
It looks like rpc.nfsd's command line syntax is all negative.
I.e. all versions are on by default and they can be turned off
using the -N option.  I can either add an option to enable minorversion
or have it enabled by default and add an option to disable it.

Benny

> 
> --b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 09:41:33PM +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 20:58 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 01:55:17PM -0400, bfields wrote:
> >> On Thu, Apr 02, 2009 at 08:52:40PM +0300, Benny Halevy wrote:
> >>> On Apr. 02, 2009, 20:45 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
> >>>> On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
> >>>>>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
> >>>>>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
> >>>>>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
> >>>>>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
> >>>>>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
> >>>>>>>>>>> On the client side minorversion 1 will be used only if the user
> >>>>>>>>>>> explicitly asked for it with mount -o minorversion=1.
> >>>>>>>>>> I'd feel more comfortable with being able to compile it out until the
> >>>>>>>>>> stability of the code has been established. I'd certainly want to be
> >>>>>>>>>> able to do that on the server side, since it has no other means to
> >>>>>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
> >>>>>>>>>> fatal condition.
> >>>>>>>>> I think it's acceptable given an interface that allows choosing the
> >>>>>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
> >>>>>>>> Is there such an interface on the server?
> >>>>>>> That's the patch Benny just posted.  It seems like a pretty simple
> >>>>>>> extension of the existing version-choosing interface
> >>>>>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
> >>>>>>> 4.1 to on?  I need to take another look.
> >>>>>> That's right.  This can be changed trivially,
> >>>>>> but I'll have to find and work the reason why the settings reset on
> >>>>>> service nfs restart, otherwise it will reset itself.
> >>>>> Maybe you're starting and stopping the server using distro init scripts
> >>>>> that load and unload the nfsd module?
> >>>> I'm using redhat's /etc/init.d/nfs
> >>>> My impression was that it doesn't unload the nfsd module when the service
> >>>> is stopped.  However it does run modprobe on "start".  I wonder if that's
> >>>> causing the module to reload.
> >>> Hmm, after "stop" nfsd is up and must stay up to keep serving
> >>> /proc/fs/nfsd/versions.
> >>>
> >>> Giving modprobe the --first-time option reports:
> >>> FATAL: Module nfsd already in kernel.
> >> Yeah, actually, /proc/fs/nfsd/versions is probably getting written
> >> somewhere in there.  That may be the effect of the --no-nfs-version
> >> mountd argument.
> > 
> > Whoops, sorry, rpc.nfsd: it should be the --no-nfs-version argument to
> > rpc.nfsd.  See nfssvc_versbits() in nfs-utils/support/nfs/nfssvc.c.
> 
> OK. I see that now.
> It looks like rpc.nfsd's command line syntax is all negative.
> I.e. all versions are on by default and they can be turned off
> using the -N option.  I can either add an option to enable minorversion
> or have it enabled by default and add an option to disable it.

I'm not a big fan of that commandline syntax--seems to me it should give
a range rather than a list of versions not to support.

For now the important thing is that the kernel default minorversion 1 to
off.  We can worry about what rpc.nfsd should do later--I guess a
commandline option to turn on specific minor versions would work....
Maybe allow --nfs-version as well as --no-nfs-version?

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 02, 2009, 21:46 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 09:41:33PM +0300, Benny Halevy wrote:
>> On Apr. 02, 2009, 20:58 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Thu, Apr 02, 2009 at 01:55:17PM -0400, bfields wrote:
>>>> On Thu, Apr 02, 2009 at 08:52:40PM +0300, Benny Halevy wrote:
>>>>> On Apr. 02, 2009, 20:45 +0300, Benny Halevy <bhalevy@panasas.com> wrote:
>>>>>> On Apr. 02, 2009, 19:54 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>> On Thu, Apr 02, 2009 at 07:34:29PM +0300, Benny Halevy wrote:
>>>>>>>> On Apr. 02, 2009, 17:31 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>>>> On Thu, Apr 02, 2009 at 10:25:55AM -0400, Trond Myklebust wrote:
>>>>>>>>>> On Thu, 2009-04-02 at 10:22 -0400, J. Bruce Fields wrote:
>>>>>>>>>>> On Thu, Apr 02, 2009 at 10:16:40AM -0400, Trond Myklebust wrote:
>>>>>>>>>>>> On Thu, 2009-04-02 at 16:46 +0300, Benny Halevy wrote:
>>>>>>>>>>>>> Trond, please speak up if you want to remove CONFIG_NFS_V4_1 as well.
>>>>>>>>>>>>> On the client side minorversion 1 will be used only if the user
>>>>>>>>>>>>> explicitly asked for it with mount -o minorversion=1.
>>>>>>>>>>>> I'd feel more comfortable with being able to compile it out until the
>>>>>>>>>>>> stability of the code has been established. I'd certainly want to be
>>>>>>>>>>>> able to do that on the server side, since it has no other means to
>>>>>>>>>>>> restrict the protocol version should it turn out that NFSv4.1 has some
>>>>>>>>>>>> fatal condition.
>>>>>>>>>>> I think it's acceptable given an interface that allows choosing the
>>>>>>>>>>> supported minorversion at runtime (and that defaults 4.1 to off).
>>>>>>>>>> Is there such an interface on the server?
>>>>>>>>> That's the patch Benny just posted.  It seems like a pretty simple
>>>>>>>>> extension of the existing version-choosing interface
>>>>>>>>> (/proc/fs/nfsd/versions), though I think the version he posted defaults
>>>>>>>>> 4.1 to on?  I need to take another look.
>>>>>>>> That's right.  This can be changed trivially,
>>>>>>>> but I'll have to find and work the reason why the settings reset on
>>>>>>>> service nfs restart, otherwise it will reset itself.
>>>>>>> Maybe you're starting and stopping the server using distro init scripts
>>>>>>> that load and unload the nfsd module?
>>>>>> I'm using redhat's /etc/init.d/nfs
>>>>>> My impression was that it doesn't unload the nfsd module when the service
>>>>>> is stopped.  However it does run modprobe on "start".  I wonder if that's
>>>>>> causing the module to reload.
>>>>> Hmm, after "stop" nfsd is up and must stay up to keep serving
>>>>> /proc/fs/nfsd/versions.
>>>>>
>>>>> Giving modprobe the --first-time option reports:
>>>>> FATAL: Module nfsd already in kernel.
>>>> Yeah, actually, /proc/fs/nfsd/versions is probably getting written
>>>> somewhere in there.  That may be the effect of the --no-nfs-version
>>>> mountd argument.
>>> Whoops, sorry, rpc.nfsd: it should be the --no-nfs-version argument to
>>> rpc.nfsd.  See nfssvc_versbits() in nfs-utils/support/nfs/nfssvc.c.
>> OK. I see that now.
>> It looks like rpc.nfsd's command line syntax is all negative.
>> I.e. all versions are on by default and they can be turned off
>> using the -N option.  I can either add an option to enable minorversion
>> or have it enabled by default and add an option to disable it.
> 
> I'm not a big fan of that commandline syntax--seems to me it should give
> a range rather than a list of versions not to support.
> 
> For now the important thing is that the kernel default minorversion 1 to
> off.  We can worry about what rpc.nfsd should do later--I guess a
> commandline option to turn on specific minor versions would work....
> Maybe allow --nfs-version as well as --no-nfs-version?

Sounds like a good idea.
How about --nfs-versions (in plural) to correspond to /proc/fs/nfsd/versions
and then pass the whole versions string?

Benny

> 
> --b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 10:05:53PM +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 21:46 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 09:41:33PM +0300, Benny Halevy wrote:
> >> It looks like rpc.nfsd's command line syntax is all negative.
> >> I.e. all versions are on by default and they can be turned off
> >> using the -N option.  I can either add an option to enable minorversion
> >> or have it enabled by default and add an option to disable it.
> > 
> > I'm not a big fan of that commandline syntax--seems to me it should give
> > a range rather than a list of versions not to support.
> > 
> > For now the important thing is that the kernel default minorversion 1 to
> > off.  We can worry about what rpc.nfsd should do later--I guess a
> > commandline option to turn on specific minor versions would work....
> > Maybe allow --nfs-version as well as --no-nfs-version?
> 
> Sounds like a good idea.
> How about --nfs-versions (in plural) to correspond to /proc/fs/nfsd/versions
> and then pass the whole versions string?

I suppose that would work, but it seems a little inconsistent with the
current syntax, and with the way unix commandlines works in general.
Something a little less idiosyncratic would be easier to document and
remember.

--b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[Benny Halevy]

On Apr. 03, 2009, 1:49 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 10:05:53PM +0300, Benny Halevy wrote:
>> On Apr. 02, 2009, 21:46 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Thu, Apr 02, 2009 at 09:41:33PM +0300, Benny Halevy wrote:
>>>> It looks like rpc.nfsd's command line syntax is all negative.
>>>> I.e. all versions are on by default and they can be turned off
>>>> using the -N option.  I can either add an option to enable minorversion
>>>> or have it enabled by default and add an option to disable it.
>>> I'm not a big fan of that commandline syntax--seems to me it should give
>>> a range rather than a list of versions not to support.
>>>
>>> For now the important thing is that the kernel default minorversion 1 to
>>> off.  We can worry about what rpc.nfsd should do later--I guess a
>>> commandline option to turn on specific minor versions would work....
>>> Maybe allow --nfs-version as well as --no-nfs-version?
>> Sounds like a good idea.
>> How about --nfs-versions (in plural) to correspond to /proc/fs/nfsd/versions
>> and then pass the whole versions string?
> 
> I suppose that would work, but it seems a little inconsistent with the
> current syntax, and with the way unix commandlines works in general.
> Something a little less idiosyncratic would be easier to document and
> remember.

OK.  I'm trying to develop such an option but I'm having trouble
with ./configure on Fedora 9. didn't find libwrap (which exists
in /lib and /lib64) and --without-tcp-wrappers it can't find
libevent, which is in /usr/lib64.

Anything special I need to do to make the nfs utils on Fedora 9?

Benny

> 
> --b.

Re: [pnfs] [PATCH v2 06/47] nfsd41: Add Kconfig symbols for NFSv4.1

[J. Bruce Fields]

On Fri, Apr 03, 2009 at 03:11:51AM +0300, Benny Halevy wrote:
> On Apr. 03, 2009, 1:49 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 10:05:53PM +0300, Benny Halevy wrote:
> >> On Apr. 02, 2009, 21:46 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>> On Thu, Apr 02, 2009 at 09:41:33PM +0300, Benny Halevy wrote:
> >>>> It looks like rpc.nfsd's command line syntax is all negative.
> >>>> I.e. all versions are on by default and they can be turned off
> >>>> using the -N option.  I can either add an option to enable minorversion
> >>>> or have it enabled by default and add an option to disable it.
> >>> I'm not a big fan of that commandline syntax--seems to me it should give
> >>> a range rather than a list of versions not to support.
> >>>
> >>> For now the important thing is that the kernel default minorversion 1 to
> >>> off.  We can worry about what rpc.nfsd should do later--I guess a
> >>> commandline option to turn on specific minor versions would work....
> >>> Maybe allow --nfs-version as well as --no-nfs-version?
> >> Sounds like a good idea.
> >> How about --nfs-versions (in plural) to correspond to /proc/fs/nfsd/versions
> >> and then pass the whole versions string?
> > 
> > I suppose that would work, but it seems a little inconsistent with the
> > current syntax, and with the way unix commandlines works in general.
> > Something a little less idiosyncratic would be easier to document and
> > remember.
> 
> OK.  I'm trying to develop such an option but I'm having trouble
> with ./configure on Fedora 9. didn't find libwrap (which exists
> in /lib and /lib64) and --without-tcp-wrappers it can't find
> libevent, which is in /usr/lib64.
> 
> Anything special I need to do to make the nfs utils on Fedora 9?

Steved's the expert, but one approach might be just to get the source
rpm.

Obviously this doesn't have to be done before the kernel submission; all
I care about is that nobody get a server that accepts 4.1 calls by
default.

If you just need something for testing, I think you can do the job of
rpc.nfsd by hand with just:

	echo <version string> >/proc/fs/nfsd/versions
	echo <number of threads> >/proc/fs/nfsd/threads

rpc.nfsd isn't exactly the sharpest knife in the drawer.

--b.

[PATCH v2 07/47] nfsd41: define nfs41 error codes

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Define all error code present in
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: clean up error code definitions]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfs.h       |    2 --
 include/linux/nfsd/nfsd.h |   43 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/include/linux/nfs.h b/include/linux/nfs.h
index 1510a64..214d499 100644
--- a/include/linux/nfs.h
+++ b/include/linux/nfs.h
@@ -109,8 +109,6 @@
 	NFSERR_FILE_OPEN = 10046,      /*       v4 */
 	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
 	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
-
-	NFSERR_REPLAY_ME = 11001,	/* linux internal */
 };
 
 /* NFSv2 file types - beware, these are not the same in NFSv3 */
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index 54beda1..ab9616d 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -249,7 +249,44 @@ void		nfsd_lockd_shutdown(void);
 #define	nfserr_cb_path_down	cpu_to_be32(NFSERR_CB_PATH_DOWN)
 #define	nfserr_locked		cpu_to_be32(NFSERR_LOCKED)
 #define	nfserr_wrongsec		cpu_to_be32(NFSERR_WRONGSEC)
-#define	nfserr_replay_me	cpu_to_be32(NFSERR_REPLAY_ME)
+#define nfserr_badiomode		cpu_to_be32(NFS4ERR_BADIOMODE)
+#define nfserr_badlayout		cpu_to_be32(NFS4ERR_BADLAYOUT)
+#define nfserr_bad_session_digest	cpu_to_be32(NFS4ERR_BAD_SESSION_DIGEST)
+#define nfserr_badsession		cpu_to_be32(NFS4ERR_BADSESSION)
+#define nfserr_badslot			cpu_to_be32(NFS4ERR_BADSLOT)
+#define nfserr_complete_already		cpu_to_be32(NFS4ERR_COMPLETE_ALREADY)
+#define nfserr_conn_not_bound_to_session cpu_to_be32(NFS4ERR_CONN_NOT_BOUND_TO_SESSION)
+#define nfserr_deleg_already_wanted	cpu_to_be32(NFS4ERR_DELEG_ALREADY_WANTED)
+#define nfserr_back_chan_busy		cpu_to_be32(NFS4ERR_BACK_CHAN_BUSY)
+#define nfserr_layouttrylater		cpu_to_be32(NFS4ERR_LAYOUTTRYLATER)
+#define nfserr_layoutunavailable	cpu_to_be32(NFS4ERR_LAYOUTUNAVAILABLE)
+#define nfserr_nomatching_layout	cpu_to_be32(NFS4ERR_NOMATCHING_LAYOUT)
+#define nfserr_recallconflict		cpu_to_be32(NFS4ERR_RECALLCONFLICT)
+#define nfserr_unknown_layouttype	cpu_to_be32(NFS4ERR_UNKNOWN_LAYOUTTYPE)
+#define nfserr_seq_misordered		cpu_to_be32(NFS4ERR_SEQ_MISORDERED)
+#define nfserr_sequence_pos		cpu_to_be32(NFS4ERR_SEQUENCE_POS)
+#define nfserr_req_too_big		cpu_to_be32(NFS4ERR_REQ_TOO_BIG)
+#define nfserr_rep_too_big		cpu_to_be32(NFS4ERR_REP_TOO_BIG)
+#define nfserr_rep_too_big_to_cache	cpu_to_be32(NFS4ERR_REP_TOO_BIG_TO_CACHE)
+#define nfserr_retry_uncached_rep	cpu_to_be32(NFS4ERR_RETRY_UNCACHED_REP)
+#define nfserr_unsafe_compound		cpu_to_be32(NFS4ERR_UNSAFE_COMPOUND)
+#define nfserr_too_many_ops		cpu_to_be32(NFS4ERR_TOO_MANY_OPS)
+#define nfserr_op_not_in_session	cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION)
+#define nfserr_hash_alg_unsupp		cpu_to_be32(NFS4ERR_HASH_ALG_UNSUPP)
+#define nfserr_clientid_busy		cpu_to_be32(NFS4ERR_CLIENTID_BUSY)
+#define nfserr_pnfs_io_hole		cpu_to_be32(NFS4ERR_PNFS_IO_HOLE)
+#define nfserr_seq_false_retry		cpu_to_be32(NFS4ERR_SEQ_FALSE_RETRY)
+#define nfserr_bad_high_slot		cpu_to_be32(NFS4ERR_BAD_HIGH_SLOT)
+#define nfserr_deadsession		cpu_to_be32(NFS4ERR_DEADSESSION)
+#define nfserr_encr_alg_unsupp		cpu_to_be32(NFS4ERR_ENCR_ALG_UNSUPP)
+#define nfserr_pnfs_no_layout		cpu_to_be32(NFS4ERR_PNFS_NO_LAYOUT)
+#define nfserr_not_only_op		cpu_to_be32(NFS4ERR_NOT_ONLY_OP)
+#define nfserr_wrong_cred		cpu_to_be32(NFS4ERR_WRONG_CRED)
+#define nfserr_wrong_type		cpu_to_be32(NFS4ERR_WRONG_TYPE)
+#define nfserr_dirdeleg_unavail		cpu_to_be32(NFS4ERR_DIRDELEG_UNAVAIL)
+#define nfserr_reject_deleg		cpu_to_be32(NFS4ERR_REJECT_DELEG)
+#define nfserr_returnconflict		cpu_to_be32(NFS4ERR_RETURNCONFLICT)
+#define nfserr_deleg_revoked		cpu_to_be32(NFS4ERR_DELEG_REVOKED)
 
 /* error codes for internal use */
 /* if a request fails due to kmalloc failure, it gets dropped.
@@ -258,6 +295,10 @@ void		nfsd_lockd_shutdown(void);
 #define	nfserr_dropit		cpu_to_be32(30000)
 /* end-of-file indicator in readdir */
 #define	nfserr_eof		cpu_to_be32(30001)
+/* replay detected */
+#define	nfserr_replay_me	cpu_to_be32(11001)
+/* nfs41 replay detected */
+#define	nfserr_replay_cache	cpu_to_be32(11002)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)	(l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
-- 
1.6.2.1

Re: [PATCH v2 07/47] nfsd41: define nfs41 error codes

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:31:15AM +0300, Benny Halevy wrote:
> From: Marc Eshel <eshel@almaden.ibm.com>
> 
> Define all error code present in
> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: clean up error code definitions]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  include/linux/nfs.h       |    2 --
>  include/linux/nfsd/nfsd.h |   43 ++++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 42 insertions(+), 3 deletions(-)
> 
> diff --git a/include/linux/nfs.h b/include/linux/nfs.h
> index 1510a64..214d499 100644
> --- a/include/linux/nfs.h
> +++ b/include/linux/nfs.h
> @@ -109,8 +109,6 @@
>  	NFSERR_FILE_OPEN = 10046,      /*       v4 */
>  	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
>  	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
> -
> -	NFSERR_REPLAY_ME = 11001,	/* linux internal */

This renders "[PATCH v2 05/47] nfsd41: change NFSERR_REPLAY_ME" moot;
just drop that previous patch.

--b.

>  };
>  
>  /* NFSv2 file types - beware, these are not the same in NFSv3 */
> diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
> index 54beda1..ab9616d 100644
> --- a/include/linux/nfsd/nfsd.h
> +++ b/include/linux/nfsd/nfsd.h
> @@ -249,7 +249,44 @@ void		nfsd_lockd_shutdown(void);
>  #define	nfserr_cb_path_down	cpu_to_be32(NFSERR_CB_PATH_DOWN)
>  #define	nfserr_locked		cpu_to_be32(NFSERR_LOCKED)
>  #define	nfserr_wrongsec		cpu_to_be32(NFSERR_WRONGSEC)
> -#define	nfserr_replay_me	cpu_to_be32(NFSERR_REPLAY_ME)
> +#define nfserr_badiomode		cpu_to_be32(NFS4ERR_BADIOMODE)
> +#define nfserr_badlayout		cpu_to_be32(NFS4ERR_BADLAYOUT)
> +#define nfserr_bad_session_digest	cpu_to_be32(NFS4ERR_BAD_SESSION_DIGEST)
> +#define nfserr_badsession		cpu_to_be32(NFS4ERR_BADSESSION)
> +#define nfserr_badslot			cpu_to_be32(NFS4ERR_BADSLOT)
> +#define nfserr_complete_already		cpu_to_be32(NFS4ERR_COMPLETE_ALREADY)
> +#define nfserr_conn_not_bound_to_session cpu_to_be32(NFS4ERR_CONN_NOT_BOUND_TO_SESSION)
> +#define nfserr_deleg_already_wanted	cpu_to_be32(NFS4ERR_DELEG_ALREADY_WANTED)
> +#define nfserr_back_chan_busy		cpu_to_be32(NFS4ERR_BACK_CHAN_BUSY)
> +#define nfserr_layouttrylater		cpu_to_be32(NFS4ERR_LAYOUTTRYLATER)
> +#define nfserr_layoutunavailable	cpu_to_be32(NFS4ERR_LAYOUTUNAVAILABLE)
> +#define nfserr_nomatching_layout	cpu_to_be32(NFS4ERR_NOMATCHING_LAYOUT)
> +#define nfserr_recallconflict		cpu_to_be32(NFS4ERR_RECALLCONFLICT)
> +#define nfserr_unknown_layouttype	cpu_to_be32(NFS4ERR_UNKNOWN_LAYOUTTYPE)
> +#define nfserr_seq_misordered		cpu_to_be32(NFS4ERR_SEQ_MISORDERED)
> +#define nfserr_sequence_pos		cpu_to_be32(NFS4ERR_SEQUENCE_POS)
> +#define nfserr_req_too_big		cpu_to_be32(NFS4ERR_REQ_TOO_BIG)
> +#define nfserr_rep_too_big		cpu_to_be32(NFS4ERR_REP_TOO_BIG)
> +#define nfserr_rep_too_big_to_cache	cpu_to_be32(NFS4ERR_REP_TOO_BIG_TO_CACHE)
> +#define nfserr_retry_uncached_rep	cpu_to_be32(NFS4ERR_RETRY_UNCACHED_REP)
> +#define nfserr_unsafe_compound		cpu_to_be32(NFS4ERR_UNSAFE_COMPOUND)
> +#define nfserr_too_many_ops		cpu_to_be32(NFS4ERR_TOO_MANY_OPS)
> +#define nfserr_op_not_in_session	cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION)
> +#define nfserr_hash_alg_unsupp		cpu_to_be32(NFS4ERR_HASH_ALG_UNSUPP)
> +#define nfserr_clientid_busy		cpu_to_be32(NFS4ERR_CLIENTID_BUSY)
> +#define nfserr_pnfs_io_hole		cpu_to_be32(NFS4ERR_PNFS_IO_HOLE)
> +#define nfserr_seq_false_retry		cpu_to_be32(NFS4ERR_SEQ_FALSE_RETRY)
> +#define nfserr_bad_high_slot		cpu_to_be32(NFS4ERR_BAD_HIGH_SLOT)
> +#define nfserr_deadsession		cpu_to_be32(NFS4ERR_DEADSESSION)
> +#define nfserr_encr_alg_unsupp		cpu_to_be32(NFS4ERR_ENCR_ALG_UNSUPP)
> +#define nfserr_pnfs_no_layout		cpu_to_be32(NFS4ERR_PNFS_NO_LAYOUT)
> +#define nfserr_not_only_op		cpu_to_be32(NFS4ERR_NOT_ONLY_OP)
> +#define nfserr_wrong_cred		cpu_to_be32(NFS4ERR_WRONG_CRED)
> +#define nfserr_wrong_type		cpu_to_be32(NFS4ERR_WRONG_TYPE)
> +#define nfserr_dirdeleg_unavail		cpu_to_be32(NFS4ERR_DIRDELEG_UNAVAIL)
> +#define nfserr_reject_deleg		cpu_to_be32(NFS4ERR_REJECT_DELEG)
> +#define nfserr_returnconflict		cpu_to_be32(NFS4ERR_RETURNCONFLICT)
> +#define nfserr_deleg_revoked		cpu_to_be32(NFS4ERR_DELEG_REVOKED)
>  
>  /* error codes for internal use */
>  /* if a request fails due to kmalloc failure, it gets dropped.
> @@ -258,6 +295,10 @@ void		nfsd_lockd_shutdown(void);
>  #define	nfserr_dropit		cpu_to_be32(30000)
>  /* end-of-file indicator in readdir */
>  #define	nfserr_eof		cpu_to_be32(30001)
> +/* replay detected */
> +#define	nfserr_replay_me	cpu_to_be32(11001)
> +/* nfs41 replay detected */
> +#define	nfserr_replay_cache	cpu_to_be32(11002)
>  
>  /* Check for dir entries '.' and '..' */
>  #define isdotent(n, l)	(l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 07/47] nfsd41: define nfs41 error codes

[Benny Halevy]

On Mar. 30, 2009, 21:16 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:31:15AM +0300, Benny Halevy wrote:
>> From: Marc Eshel <eshel@almaden.ibm.com>
>>
>> Define all error code present in
>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29.
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfsd41: clean up error code definitions]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  include/linux/nfs.h       |    2 --
>>  include/linux/nfsd/nfsd.h |   43 ++++++++++++++++++++++++++++++++++++++++++-
>>  2 files changed, 42 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/nfs.h b/include/linux/nfs.h
>> index 1510a64..214d499 100644
>> --- a/include/linux/nfs.h
>> +++ b/include/linux/nfs.h
>> @@ -109,8 +109,6 @@
>>  	NFSERR_FILE_OPEN = 10046,      /*       v4 */
>>  	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
>>  	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
>> -
>> -	NFSERR_REPLAY_ME = 11001,	/* linux internal */
> 
> This renders "[PATCH v2 05/47] nfsd41: change NFSERR_REPLAY_ME" moot;
> just drop that previous patch.

I see you already pulled the former patch into for-2.6.30...

Benny

> 
> --b.

Re: [PATCH v2 07/47] nfsd41: define nfs41 error codes

[J. Bruce Fields]

On Mon, Mar 30, 2009 at 10:59:23PM +0300, Benny Halevy wrote:
> On Mar. 30, 2009, 21:16 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:31:15AM +0300, Benny Halevy wrote:
> >> From: Marc Eshel <eshel@almaden.ibm.com>
> >>
> >> Define all error code present in
> >> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29.
> >>
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> [nfsd41: clean up error code definitions]
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> ---
> >>  include/linux/nfs.h       |    2 --
> >>  include/linux/nfsd/nfsd.h |   43 ++++++++++++++++++++++++++++++++++++++++++-
> >>  2 files changed, 42 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/include/linux/nfs.h b/include/linux/nfs.h
> >> index 1510a64..214d499 100644
> >> --- a/include/linux/nfs.h
> >> +++ b/include/linux/nfs.h
> >> @@ -109,8 +109,6 @@
> >>  	NFSERR_FILE_OPEN = 10046,      /*       v4 */
> >>  	NFSERR_ADMIN_REVOKED = 10047,  /*       v4 */
> >>  	NFSERR_CB_PATH_DOWN = 10048,   /*       v4 */
> >> -
> >> -	NFSERR_REPLAY_ME = 11001,	/* linux internal */
> > 
> > This renders "[PATCH v2 05/47] nfsd41: change NFSERR_REPLAY_ME" moot;
> > just drop that previous patch.
> 
> I see you already pulled the former patch into for-2.6.30...

Yes.  Should be fixed now, though.

--b.

[PATCH v2 08/47] nfsd41: sessions basic data types

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

This patch provides basic data structures representing the nfs41
sessions and slots, plus helpers for keeping a reference count
on the session and freeing it.

Note that our server only support a headerpadsz of 0 and
it ignores backchannel attributes at the moment.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: remove headerpadsz from channel attributes]
[nfsd41: embed nfsd4_channel in nfsd4_session]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |   22 ++++++++++++++++++++++
 include/linux/nfsd/state.h |   34 ++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 070e9e5..f4b624b 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -382,6 +382,28 @@ static void release_openowner(struct nfs4_stateowner *sop)
 	nfs4_put_stateowner(sop);
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+static void
+release_session(struct nfsd4_session *ses)
+{
+	list_del(&ses->se_hash);
+	list_del(&ses->se_perclnt);
+	nfsd4_put_session(ses);
+}
+
+void
+free_session(struct kref *kref)
+{
+	struct nfsd4_session *ses;
+
+	ses = container_of(kref, struct nfsd4_session, se_ref);
+	kfree(ses->se_slots);
+	kfree(ses);
+}
+#else /* CONFIG_NFSD_V4_1 */
+static inline void release_session(struct nfsd4_session *ses) {}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static inline void
 renew_client(struct nfs4_client *clp)
 {
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index a6e4a00..afae103 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -99,6 +99,40 @@ struct nfs4_callback {
 	struct rpc_clnt *       cb_client;
 };
 
+struct nfsd4_slot {
+	bool				sl_inuse;
+	struct nfsd4_session		*sl_session;
+	u32				sl_seqid;
+};
+
+struct nfsd4_session {
+	struct kref		se_ref;
+	struct list_head	se_hash;	/* hash by sessionid */
+	struct list_head	se_perclnt;
+	u32			se_flags;
+	struct nfs4_client	*se_client;	/* for expire_client */
+	struct nfs4_sessionid	se_sessionid;
+	u32			se_fmaxreq_sz;
+	u32			se_fmaxresp_sz;
+	u32			se_fmaxresp_cached;
+	u32			se_fmaxops;
+	u32			se_fnumslots;
+	struct nfsd4_slot	*se_slots;	/* forward channel slots */
+};
+
+static inline void
+nfsd4_put_session(struct nfsd4_session *ses)
+{
+	extern void free_session(struct kref *kref);
+	kref_put(&ses->se_ref, free_session);
+}
+
+static inline void
+nfsd4_get_session(struct nfsd4_session *ses)
+{
+	kref_get(&ses->se_ref);
+}
+
 #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
 
 /*
-- 
1.6.2.1

[PATCH v2 09/47] nfsd41: introduce nfs4_client cl_sessions list

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |    3 +++
 include/linux/nfsd/state.h |    3 +++
 2 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f4b624b..b71ec72 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -532,6 +532,9 @@ static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
 	INIT_LIST_HEAD(&clp->cl_strhash);
 	INIT_LIST_HEAD(&clp->cl_openowners);
 	INIT_LIST_HEAD(&clp->cl_delegations);
+#if defined(CONFIG_NFSD_V4_1)
+	INIT_LIST_HEAD(&clp->cl_sessions);
+#endif /* CONFIG_NFSD_V4_1 */
 	INIT_LIST_HEAD(&clp->cl_lru);
 	return clp;
 }
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index afae103..29624b4 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -164,6 +164,9 @@ struct nfs4_client {
 	struct nfs4_callback	cl_callback;    /* callback info */
 	atomic_t		cl_count;	/* ref count */
 	u32			cl_firststate;	/* recovery dir creation */
+#ifdef CONFIG_NFSD_V4_1
+	struct list_head	cl_sessions;
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 /* struct nfs4_client_reset
-- 
1.6.2.1

[PATCH v2 10/47] nfsd41: release_session when client is expired

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[add CONFIG_NFSD_V4_1 to fix v4.0 regression bug]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index b71ec72..df9d42e 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -515,6 +515,14 @@ expire_client(struct nfs4_client *clp)
 		sop = list_entry(clp->cl_openowners.next, struct nfs4_stateowner, so_perclient);
 		release_openowner(sop);
 	}
+#ifdef CONFIG_NFSD_V4_1
+	while (!list_empty(&clp->cl_sessions)) {
+		struct nfsd4_session  *ses;
+		ses = list_entry(clp->cl_sessions.next, struct nfsd4_session,
+				 se_perclnt);
+		release_session(ses);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 	put_nfs4_client(clp);
 }
 
-- 
1.6.2.1

[PATCH v2 11/47] nfsd41: sessionid hashing

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Simple sessionid hashing using its monotonically increasing sequence number.

Locking considerations:
sessionid_hashtbl access is controlled by the sessionid_lock spin lock.
It must be taken for insert, delete, and lookup.
nfsd4_sequence looks up the session id and if the session is found,
it calls nfsd4_get_session (still under the sessionid_lock).
nfsd4_destroy_session calls nfsd4_put_session after unhashing
it, so when the session's kref reaches zero it's going to get freed.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[we don't use a prime for sessionid hash table size]
[use sessionid_lock spin lock]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |   57 +++++++++++++++++++++++++++++++++++++++++++-
 include/linux/nfsd/state.h |    7 +++++
 2 files changed, 63 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index df9d42e..ac4e8f2 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -383,11 +383,62 @@ static void release_openowner(struct nfs4_stateowner *sop)
 }
 
 #if defined(CONFIG_NFSD_V4_1)
+static DEFINE_SPINLOCK(sessionid_lock);
+#define SESSION_HASH_SIZE	512
+static struct list_head sessionid_hashtbl[SESSION_HASH_SIZE];
+
+static inline int
+hash_sessionid(struct nfs4_sessionid *sessionid)
+{
+	struct nfsd4_sessionid *sid = (struct nfsd4_sessionid *)sessionid;
+
+	return sid->sequence % SESSION_HASH_SIZE;
+}
+
+static inline void
+dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
+{
+	u32 *ptr = (u32 *)(&sessionid->data[0]);
+	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
+}
+
+/* caller must hold sessionid_lock */
+static struct nfsd4_session *
+find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
+{
+	struct nfsd4_session *elem;
+	int idx;
+
+	dump_sessionid(__func__, sessionid);
+	idx = hash_sessionid(sessionid);
+	dprintk("%s: idx is %d\n", __func__, idx);
+	/* Search in the appropriate list */
+	list_for_each_entry(elem, &sessionid_hashtbl[idx], se_hash) {
+		dump_sessionid("list traversal", &elem->se_sessionid);
+		if (!memcmp(elem->se_sessionid.data, sessionid->data,
+			    NFS4_MAX_SESSIONID_LEN)) {
+			return elem;
+		}
+	}
+
+	dprintk("%s: session not found\n", __func__);
+	return NULL;
+}
+
+/* caller must hold sessionid_lock */
 static void
-release_session(struct nfsd4_session *ses)
+unhash_session(struct nfsd4_session *ses)
 {
 	list_del(&ses->se_hash);
 	list_del(&ses->se_perclnt);
+}
+
+static void
+release_session(struct nfsd4_session *ses)
+{
+	spin_lock(&sessionid_lock);
+	unhash_session(ses);
+	spin_unlock(&sessionid_lock);
 	nfsd4_put_session(ses);
 }
 
@@ -3213,6 +3264,10 @@ nfs4_state_init(void)
 		INIT_LIST_HEAD(&unconf_str_hashtbl[i]);
 		INIT_LIST_HEAD(&unconf_id_hashtbl[i]);
 	}
+#if defined(CONFIG_NFSD_V4_1)
+	for (i = 0; i < SESSION_HASH_SIZE; i++)
+		INIT_LIST_HEAD(&sessionid_hashtbl[i]);
+#endif /* CONFIG_NFSD_V4_1 */
 	for (i = 0; i < FILE_HASH_SIZE; i++) {
 		INIT_LIST_HEAD(&file_hashtbl[i]);
 	}
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 29624b4..7592d7b 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -133,6 +133,13 @@ nfsd4_get_session(struct nfsd4_session *ses)
 	kref_get(&ses->se_ref);
 }
 
+/* formatted contents of nfs4_sessionid */
+struct nfsd4_sessionid {
+	clientid_t	clientid;
+	u32		sequence;
+	u32		reserved;
+};
+
 #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
 
 /*
-- 
1.6.2.1

Re: [PATCH v2 11/47] nfsd41: sessionid hashing

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:31:50AM +0300, Benny Halevy wrote:
> From: Marc Eshel <eshel@almaden.ibm.com>
> 
> Simple sessionid hashing using its monotonically increasing sequence number.
> 
> Locking considerations:
> sessionid_hashtbl access is controlled by the sessionid_lock spin lock.
> It must be taken for insert, delete, and lookup.
> nfsd4_sequence looks up the session id and if the session is found,
> it calls nfsd4_get_session (still under the sessionid_lock).
> nfsd4_destroy_session calls nfsd4_put_session after unhashing
> it, so when the session's kref reaches zero it's going to get freed.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [we don't use a prime for sessionid hash table size]
> [use sessionid_lock spin lock]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |   57 +++++++++++++++++++++++++++++++++++++++++++-
>  include/linux/nfsd/state.h |    7 +++++
>  2 files changed, 63 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index df9d42e..ac4e8f2 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -383,11 +383,62 @@ static void release_openowner(struct nfs4_stateowner *sop)
>  }
>  
>  #if defined(CONFIG_NFSD_V4_1)
> +static DEFINE_SPINLOCK(sessionid_lock);
> +#define SESSION_HASH_SIZE	512
> +static struct list_head sessionid_hashtbl[SESSION_HASH_SIZE];
> +
> +static inline int
> +hash_sessionid(struct nfs4_sessionid *sessionid)
> +{
> +	struct nfsd4_sessionid *sid = (struct nfsd4_sessionid *)sessionid;
> +
> +	return sid->sequence % SESSION_HASH_SIZE;
> +}
> +
> +static inline void
> +dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
> +{
> +	u32 *ptr = (u32 *)(&sessionid->data[0]);
> +	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
> +}
> +
> +/* caller must hold sessionid_lock */
> +static struct nfsd4_session *
> +find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
> +{
> +	struct nfsd4_session *elem;
> +	int idx;
> +
> +	dump_sessionid(__func__, sessionid);
> +	idx = hash_sessionid(sessionid);
> +	dprintk("%s: idx is %d\n", __func__, idx);
> +	/* Search in the appropriate list */
> +	list_for_each_entry(elem, &sessionid_hashtbl[idx], se_hash) {
> +		dump_sessionid("list traversal", &elem->se_sessionid);
> +		if (!memcmp(elem->se_sessionid.data, sessionid->data,
> +			    NFS4_MAX_SESSIONID_LEN)) {
> +			return elem;
> +		}
> +	}
> +
> +	dprintk("%s: session not found\n", __func__);

Massive dprintk overkill in this function.

> +	return NULL;
> +}
> +
> +/* caller must hold sessionid_lock */
>  static void
> -release_session(struct nfsd4_session *ses)
> +unhash_session(struct nfsd4_session *ses)
>  {
>  	list_del(&ses->se_hash);
>  	list_del(&ses->se_perclnt);
> +}
> +
> +static void
> +release_session(struct nfsd4_session *ses)
> +{
> +	spin_lock(&sessionid_lock);
> +	unhash_session(ses);
> +	spin_unlock(&sessionid_lock);
>  	nfsd4_put_session(ses);
>  }

It's not obvious from the names what the difference between
release_session() and nfsd4_put_session() is.

How about just renaming release_session() to unhash_session(), and
dumping hash_session?  The two list_del()'s don't need their own
function.

--b.

>  
> @@ -3213,6 +3264,10 @@ nfs4_state_init(void)
>  		INIT_LIST_HEAD(&unconf_str_hashtbl[i]);
>  		INIT_LIST_HEAD(&unconf_id_hashtbl[i]);
>  	}
> +#if defined(CONFIG_NFSD_V4_1)
> +	for (i = 0; i < SESSION_HASH_SIZE; i++)
> +		INIT_LIST_HEAD(&sessionid_hashtbl[i]);
> +#endif /* CONFIG_NFSD_V4_1 */
>  	for (i = 0; i < FILE_HASH_SIZE; i++) {
>  		INIT_LIST_HEAD(&file_hashtbl[i]);
>  	}
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 29624b4..7592d7b 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -133,6 +133,13 @@ nfsd4_get_session(struct nfsd4_session *ses)
>  	kref_get(&ses->se_ref);
>  }
>  
> +/* formatted contents of nfs4_sessionid */
> +struct nfsd4_sessionid {
> +	clientid_t	clientid;
> +	u32		sequence;
> +	u32		reserved;
> +};
> +
>  #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
>  
>  /*
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 11/47] nfsd41: sessionid hashing

[Benny Halevy]

On Mar. 30, 2009, 23:08 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:31:50AM +0300, Benny Halevy wrote:
>> From: Marc Eshel <eshel@almaden.ibm.com>
>>
>> Simple sessionid hashing using its monotonically increasing sequence number.
>>
>> Locking considerations:
>> sessionid_hashtbl access is controlled by the sessionid_lock spin lock.
>> It must be taken for insert, delete, and lookup.
>> nfsd4_sequence looks up the session id and if the session is found,
>> it calls nfsd4_get_session (still under the sessionid_lock).
>> nfsd4_destroy_session calls nfsd4_put_session after unhashing
>> it, so when the session's kref reaches zero it's going to get freed.
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [we don't use a prime for sessionid hash table size]
>> [use sessionid_lock spin lock]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4state.c        |   57 +++++++++++++++++++++++++++++++++++++++++++-
>>  include/linux/nfsd/state.h |    7 +++++
>>  2 files changed, 63 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index df9d42e..ac4e8f2 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -383,11 +383,62 @@ static void release_openowner(struct nfs4_stateowner *sop)
>>  }
>>  
>>  #if defined(CONFIG_NFSD_V4_1)
>> +static DEFINE_SPINLOCK(sessionid_lock);
>> +#define SESSION_HASH_SIZE	512
>> +static struct list_head sessionid_hashtbl[SESSION_HASH_SIZE];
>> +
>> +static inline int
>> +hash_sessionid(struct nfs4_sessionid *sessionid)
>> +{
>> +	struct nfsd4_sessionid *sid = (struct nfsd4_sessionid *)sessionid;
>> +
>> +	return sid->sequence % SESSION_HASH_SIZE;
>> +}
>> +
>> +static inline void
>> +dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
>> +{
>> +	u32 *ptr = (u32 *)(&sessionid->data[0]);
>> +	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
>> +}
>> +
>> +/* caller must hold sessionid_lock */
>> +static struct nfsd4_session *
>> +find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
>> +{
>> +	struct nfsd4_session *elem;
>> +	int idx;
>> +
>> +	dump_sessionid(__func__, sessionid);
>> +	idx = hash_sessionid(sessionid);
>> +	dprintk("%s: idx is %d\n", __func__, idx);
>> +	/* Search in the appropriate list */
>> +	list_for_each_entry(elem, &sessionid_hashtbl[idx], se_hash) {
>> +		dump_sessionid("list traversal", &elem->se_sessionid);
>> +		if (!memcmp(elem->se_sessionid.data, sessionid->data,
>> +			    NFS4_MAX_SESSIONID_LEN)) {
>> +			return elem;
>> +		}
>> +	}
>> +
>> +	dprintk("%s: session not found\n", __func__);
> 
> Massive dprintk overkill in this function.

Yeah, I agree we can get rid of them by now.

> 
>> +	return NULL;
>> +}
>> +
>> +/* caller must hold sessionid_lock */
>>  static void
>> -release_session(struct nfsd4_session *ses)
>> +unhash_session(struct nfsd4_session *ses)
>>  {
>>  	list_del(&ses->se_hash);
>>  	list_del(&ses->se_perclnt);
>> +}
>> +
>> +static void
>> +release_session(struct nfsd4_session *ses)
>> +{
>> +	spin_lock(&sessionid_lock);
>> +	unhash_session(ses);
>> +	spin_unlock(&sessionid_lock);
>>  	nfsd4_put_session(ses);
>>  }
> 
> It's not obvious from the names what the difference between
> release_session() and nfsd4_put_session() is.
> 
> How about just renaming release_session() to unhash_session(), and
> dumping hash_session?  The two list_del()'s don't need their own
> function.

We call unhash_session on its own later on from destroy_session,
then we destroy the callback client and finally put the session.

We can embed release_session into expire_client since it's
its only use though expire_client is hairy enough I'm not
sure we want to add more stuff into it.  If we're going
this direction, I'd consider refactoring it and taking
the many loops it's doing out into their own functions.
(we'll add a couple more for pNFS - for releasing layouts
and layoutrecalls)

Benny

> 
> --b.
> 
>>  
>> @@ -3213,6 +3264,10 @@ nfs4_state_init(void)
>>  		INIT_LIST_HEAD(&unconf_str_hashtbl[i]);
>>  		INIT_LIST_HEAD(&unconf_id_hashtbl[i]);
>>  	}
>> +#if defined(CONFIG_NFSD_V4_1)
>> +	for (i = 0; i < SESSION_HASH_SIZE; i++)
>> +		INIT_LIST_HEAD(&sessionid_hashtbl[i]);
>> +#endif /* CONFIG_NFSD_V4_1 */
>>  	for (i = 0; i < FILE_HASH_SIZE; i++) {
>>  		INIT_LIST_HEAD(&file_hashtbl[i]);
>>  	}
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index 29624b4..7592d7b 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -133,6 +133,13 @@ nfsd4_get_session(struct nfsd4_session *ses)
>>  	kref_get(&ses->se_ref);
>>  }
>>  
>> +/* formatted contents of nfs4_sessionid */
>> +struct nfsd4_sessionid {
>> +	clientid_t	clientid;
>> +	u32		sequence;
>> +	u32		reserved;
>> +};
>> +
>>  #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
>>  
>>  /*
>> -- 
>> 1.6.2.1
>>

Re: [PATCH v2 11/47] nfsd41: sessionid hashing

[J. Bruce Fields]

On Mon, Mar 30, 2009 at 11:34:32PM +0300, Benny Halevy wrote:
> On Mar. 30, 2009, 23:08 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:31:50AM +0300, Benny Halevy wrote:
> >> +	return NULL;
> >> +}
> >> +
> >> +/* caller must hold sessionid_lock */
> >>  static void
> >> -release_session(struct nfsd4_session *ses)
> >> +unhash_session(struct nfsd4_session *ses)
> >>  {
> >>  	list_del(&ses->se_hash);
> >>  	list_del(&ses->se_perclnt);
> >> +}
> >> +
> >> +static void
> >> +release_session(struct nfsd4_session *ses)
> >> +{
> >> +	spin_lock(&sessionid_lock);
> >> +	unhash_session(ses);
> >> +	spin_unlock(&sessionid_lock);
> >>  	nfsd4_put_session(ses);
> >>  }
> > 
> > It's not obvious from the names what the difference between
> > release_session() and nfsd4_put_session() is.
> > 
> > How about just renaming release_session() to unhash_session(), and
> > dumping hash_session?  The two list_del()'s don't need their own
> > function.
> 
> We call unhash_session on its own later on from destroy_session,
> then we destroy the callback client and finally put the session.

Oops, I see, I missed that.

> 
> We can embed release_session into expire_client since it's
> its only use though expire_client is hairy enough I'm not
> sure we want to add more stuff into it.  If we're going
> this direction, I'd consider refactoring it and taking
> the many loops it's doing out into their own functions.
> (we'll add a couple more for pNFS - for releasing layouts
> and layoutrecalls)

Sounds OK.--b.

Re: [PATCH v2 11/47] nfsd41: sessionid hashing

[Benny Halevy]

On Mar. 30, 2009, 23:59 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Mon, Mar 30, 2009 at 11:34:32PM +0300, Benny Halevy wrote:
>> On Mar. 30, 2009, 23:08 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Sat, Mar 28, 2009 at 11:31:50AM +0300, Benny Halevy wrote:
>>>> +	return NULL;
>>>> +}
>>>> +
>>>> +/* caller must hold sessionid_lock */
>>>>  static void
>>>> -release_session(struct nfsd4_session *ses)
>>>> +unhash_session(struct nfsd4_session *ses)
>>>>  {
>>>>  	list_del(&ses->se_hash);
>>>>  	list_del(&ses->se_perclnt);
>>>> +}
>>>> +
>>>> +static void
>>>> +release_session(struct nfsd4_session *ses)
>>>> +{
>>>> +	spin_lock(&sessionid_lock);
>>>> +	unhash_session(ses);
>>>> +	spin_unlock(&sessionid_lock);
>>>>  	nfsd4_put_session(ses);
>>>>  }
>>> It's not obvious from the names what the difference between
>>> release_session() and nfsd4_put_session() is.
>>>
>>> How about just renaming release_session() to unhash_session(), and
>>> dumping hash_session?  The two list_del()'s don't need their own
>>> function.
>> We call unhash_session on its own later on from destroy_session,
>> then we destroy the callback client and finally put the session.
> 
> Oops, I see, I missed that.
> 
>> We can embed release_session into expire_client since it's
>> its only use though expire_client is hairy enough I'm not
>> sure we want to add more stuff into it.  If we're going
>> this direction, I'd consider refactoring it and taking
>> the many loops it's doing out into their own functions.
>> (we'll add a couple more for pNFS - for releasing layouts
>> and layoutrecalls)
> 
> Sounds OK.--b.

OK.  I'll send a cleanup patch that will bring the code into
the final version.  Eventually it'll split into one patch against
for-2.6.30 to refactor expire_client and the rest will be squashed
here. (plus the changes will percolate through to the pnfs branch)

Benny

[PATCH v2 12/47] nfsd41: xdr infrastructure

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Define nfsd41_dec_ops vector and add it to nfsd4_minorversion for
minorversion 1.

Note: nfsd4_enc_ops vector is shared for v4.0 and v4.1
since we don't need to filter out obsolete ops as this is
done in the decoding phase.

exchange_id, create_session, destroy_session, and sequence ops are
implemented as stubs returning nfserr_opnotsupp at this stage.

[was nfsd41: xdr stubs]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c         |  156 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfsd/xdr4.h |   24 +++++++
 2 files changed, 180 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 76a0b2a..b082d07 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -996,6 +996,36 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
 	DECODE_TAIL;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+static __be32
+nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
+			 struct nfsd4_exchange_id *clid)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+
+static __be32
+nfsd4_decode_create_session(struct nfsd4_compoundargs *argp,
+			    struct nfsd4_create_session *sess)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+
+static __be32
+nfsd4_decode_destroy_session(struct nfsd4_compoundargs *argp,
+			     struct nfsd4_destroy_session *destroy_session)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+
+static __be32
+nfsd4_decode_sequence(struct nfsd4_compoundargs *argp,
+		      struct nfsd4_sequence *seq)
+{
+	return nfserr_opnotsupp;	/* stub */
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static __be32
 nfsd4_decode_noop(struct nfsd4_compoundargs *argp, void *p)
 {
@@ -1050,6 +1080,69 @@ static nfsd4_dec nfsd4_dec_ops[] = {
 	[OP_RELEASE_LOCKOWNER]	= (nfsd4_dec)nfsd4_decode_release_lockowner,
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+static nfsd4_dec nfsd41_dec_ops[] = {
+	[OP_ACCESS]		(nfsd4_dec)nfsd4_decode_access,
+	[OP_CLOSE]		(nfsd4_dec)nfsd4_decode_close,
+	[OP_COMMIT]		(nfsd4_dec)nfsd4_decode_commit,
+	[OP_CREATE]		(nfsd4_dec)nfsd4_decode_create,
+	[OP_DELEGPURGE]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_DELEGRETURN]	(nfsd4_dec)nfsd4_decode_delegreturn,
+	[OP_GETATTR]		(nfsd4_dec)nfsd4_decode_getattr,
+	[OP_GETFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_LINK]		(nfsd4_dec)nfsd4_decode_link,
+	[OP_LOCK]		(nfsd4_dec)nfsd4_decode_lock,
+	[OP_LOCKT]		(nfsd4_dec)nfsd4_decode_lockt,
+	[OP_LOCKU]		(nfsd4_dec)nfsd4_decode_locku,
+	[OP_LOOKUP]		(nfsd4_dec)nfsd4_decode_lookup,
+	[OP_LOOKUPP]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_NVERIFY]		(nfsd4_dec)nfsd4_decode_verify,
+	[OP_OPEN]		(nfsd4_dec)nfsd4_decode_open,
+	[OP_OPENATTR]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OPEN_CONFIRM]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OPEN_DOWNGRADE]	(nfsd4_dec)nfsd4_decode_open_downgrade,
+	[OP_PUTFH]		(nfsd4_dec)nfsd4_decode_putfh,
+	[OP_PUTPUBFH]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_PUTROOTFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_READ]		(nfsd4_dec)nfsd4_decode_read,
+	[OP_READDIR]		(nfsd4_dec)nfsd4_decode_readdir,
+	[OP_READLINK]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_REMOVE]		(nfsd4_dec)nfsd4_decode_remove,
+	[OP_RENAME]		(nfsd4_dec)nfsd4_decode_rename,
+	[OP_RENEW]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_RESTOREFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_SAVEFH]		(nfsd4_dec)nfsd4_decode_noop,
+	[OP_SECINFO]		(nfsd4_dec)nfsd4_decode_secinfo,
+	[OP_SETATTR]		(nfsd4_dec)nfsd4_decode_setattr,
+	[OP_SETCLIENTID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_SETCLIENTID_CONFIRM](nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_VERIFY]		(nfsd4_dec)nfsd4_decode_verify,
+	[OP_WRITE]		(nfsd4_dec)nfsd4_decode_write,
+	[OP_RELEASE_LOCKOWNER]	(nfsd4_dec)nfsd4_decode_notsupp,
+
+	/* new operations for NFSv4.1 */
+	[OP_BACKCHANNEL_CTL]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_BIND_CONN_TO_SESSION](nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_EXCHANGE_ID]	(nfsd4_dec)nfsd4_decode_exchange_id,
+	[OP_CREATE_SESSION]	(nfsd4_dec)nfsd4_decode_create_session,
+	[OP_DESTROY_SESSION]	(nfsd4_dec)nfsd4_decode_destroy_session,
+	[OP_FREE_STATEID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_GET_DIR_DELEGATION]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_GETDEVICEINFO]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_GETDEVICELIST]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_LAYOUTCOMMIT]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_LAYOUTGET]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_LAYOUTRETURN]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_SECINFO_NO_NAME]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_SEQUENCE]		(nfsd4_dec)nfsd4_decode_sequence,
+	[OP_SET_SSV]		(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_TEST_STATEID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_WANT_DELEGATION]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_DESTROY_CLIENTID]	(nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_RECLAIM_COMPLETE]	(nfsd4_dec)nfsd4_decode_notsupp,
+};
+#endif /* CONFIG_NFSD_V4_1 */
+
 struct nfsd4_minorversion_ops {
 	nfsd4_dec *decoders;
 	int nops;
@@ -1057,6 +1150,9 @@ struct nfsd4_minorversion_ops {
 
 static struct nfsd4_minorversion_ops nfsd4_minorversion[] = {
 	[0] = { nfsd4_dec_ops, ARRAY_SIZE(nfsd4_dec_ops) },
+#if defined(CONFIG_NFSD_V4_1)
+	[1] = { nfsd41_dec_ops, ARRAY_SIZE(nfsd41_dec_ops) },
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 static __be32
@@ -2571,6 +2667,40 @@ nfsd4_encode_write(struct nfsd4_compoundres *resp, __be32 nfserr, struct nfsd4_w
 	return nfserr;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+static __be32
+nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
+			 struct nfsd4_exchange_id *exid)
+{
+	/* stub */
+	return nfserr;
+}
+
+static __be32
+nfsd4_encode_create_session(struct nfsd4_compoundres *resp, int nfserr,
+			    struct nfsd4_create_session *sess)
+{
+	/* stub */
+	return nfserr;
+}
+
+static __be32
+nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
+			     struct nfsd4_destroy_session *destroy_session)
+{
+	/* stub */
+	return nfserr;
+}
+
+static __be32
+nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
+		      struct nfsd4_sequence *seq)
+{
+	/* stub */
+	return nfserr;
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static __be32
 nfsd4_encode_noop(struct nfsd4_compoundres *resp, __be32 nfserr, void *p)
 {
@@ -2579,6 +2709,11 @@ nfsd4_encode_noop(struct nfsd4_compoundres *resp, __be32 nfserr, void *p)
 
 typedef __be32(* nfsd4_enc)(struct nfsd4_compoundres *, __be32, void *);
 
+/*
+ * Note: nfsd4_enc_ops vector is shared for v4.0 and v4.1
+ * since we don't need to filter out obsolete ops as this is
+ * done in the decoding phase.
+ */
 static nfsd4_enc nfsd4_enc_ops[] = {
 	[OP_ACCESS]		= (nfsd4_enc)nfsd4_encode_access,
 	[OP_CLOSE]		= (nfsd4_enc)nfsd4_encode_close,
@@ -2617,6 +2752,27 @@ static nfsd4_enc nfsd4_enc_ops[] = {
 	[OP_VERIFY]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_WRITE]		= (nfsd4_enc)nfsd4_encode_write,
 	[OP_RELEASE_LOCKOWNER]	= (nfsd4_enc)nfsd4_encode_noop,
+#if defined(CONFIG_NFSD_V4_1)
+	[OP_BACKCHANNEL_CTL]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_BIND_CONN_TO_SESSION] = (nfsd4_enc)nfsd4_encode_noop,
+	[OP_EXCHANGE_ID]	= (nfsd4_enc)nfsd4_encode_exchange_id,
+	[OP_CREATE_SESSION]	= (nfsd4_enc)nfsd4_encode_create_session,
+	[OP_DESTROY_SESSION]	= (nfsd4_enc)nfsd4_encode_destroy_session,
+	[OP_FREE_STATEID]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_GET_DIR_DELEGATION]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_GETDEVICEINFO]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_GETDEVICELIST]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_LAYOUTCOMMIT]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_LAYOUTGET]		= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_LAYOUTRETURN]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_SECINFO_NO_NAME]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_SEQUENCE]		= (nfsd4_enc)nfsd4_encode_sequence,
+	[OP_SET_SSV]		= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_TEST_STATEID]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_WANT_DELEGATION]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_DESTROY_CLIENTID]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_RECLAIM_COMPLETE]	= (nfsd4_enc)nfsd4_encode_noop,
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 void
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index e309d55..40b148e 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -346,6 +346,24 @@ struct nfsd4_write {
 	nfs4_verifier	wr_verifier;        /* response */
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+struct nfsd4_exchange_id {
+	int	foo;	/* stub */
+};
+
+struct nfsd4_create_session {
+	int	foo;	/* stub */
+};
+
+struct nfsd4_sequence {
+	int	foo;	/* stub */
+};
+
+struct nfsd4_destroy_session {
+	int	foo;	/* stub */
+};
+#endif /* CONFIG_NFSD_V4_1 */
+
 struct nfsd4_op {
 	int					opnum;
 	__be32					status;
@@ -380,6 +398,12 @@ struct nfsd4_op {
 		struct nfsd4_verify		verify;
 		struct nfsd4_write		write;
 		struct nfsd4_release_lockowner	release_lockowner;
+#if defined(CONFIG_NFSD_V4_1)
+		struct nfsd4_exchange_id	exchange_id;
+		struct nfsd4_create_session	create_session;
+		struct nfsd4_destroy_session	destroy_session;
+		struct nfsd4_sequence		sequence;
+#endif /* CONFIG_NFSD_V4_1 */
 	} u;
 	struct nfs4_replay *			replay;
 };
-- 
1.6.2.1

[PATCH v2 13/47] nfsd: remove nfsd4_ops array size

[Benny Halevy]

There's no need for it.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index c8e1ad0..ded469f 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -951,7 +951,7 @@ out:
 	return status;
 }
 
-static struct nfsd4_operation nfsd4_ops[OP_RELEASE_LOCKOWNER+1] = {
+static struct nfsd4_operation nfsd4_ops[] = {
 	[OP_ACCESS] = {
 		.op_func = (nfsd4op_func)nfsd4_access,
 		.op_name = "OP_ACCESS",
-- 
1.6.2.1

[PATCH v2 14/47] nfsd41: proc stubs

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   22 ++++++++++++++++++++++
 fs/nfsd/nfs4state.c       |   34 ++++++++++++++++++++++++++++++++++
 include/linux/nfsd/xdr4.h |   14 ++++++++++++++
 3 files changed, 70 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index ded469f..f618e8e 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1101,6 +1101,28 @@ static struct nfsd4_operation nfsd4_ops[] = {
 		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_ON_ABSENT_FS,
 		.op_name = "OP_RELEASE_LOCKOWNER",
 	},
+#if defined(CONFIG_NFSD_V4_1)
+	[OP_EXCHANGE_ID] = {
+		.op_func = (nfsd4op_func)nfsd4_exchange_id,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_EXCHANGE_ID",
+	},
+	[OP_CREATE_SESSION] = {
+		.op_func = (nfsd4op_func)nfsd4_create_session,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_CREATE_SESSION",
+	},
+	[OP_DESTROY_SESSION] = {
+		.op_func = (nfsd4op_func)nfsd4_destroy_session,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_DESTROY_SESSION",
+	},
+	[OP_SEQUENCE] = {
+		.op_func = (nfsd4op_func)nfsd4_sequence,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_SEQUENCE",
+	},
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 static const char *nfsd4_op_name(unsigned opnum)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index ac4e8f2..bbb7455 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -840,6 +840,40 @@ out_err:
 	return;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+__be32
+nfsd4_exchange_id(struct svc_rqst *rqstp,
+		  struct nfsd4_compound_state *cstate,
+		  struct nfsd4_exchange_id *exid)
+{
+	return -1;	/* stub */
+}
+
+__be32
+nfsd4_create_session(struct svc_rqst *rqstp,
+		     struct nfsd4_compound_state *cstate,
+		     struct nfsd4_create_session *cr_ses)
+{
+	return -1;	/* stub */
+}
+
+__be32
+nfsd4_destroy_session(struct svc_rqst *r,
+		      struct nfsd4_compound_state *cstate,
+		      struct nfsd4_destroy_session *sessionid)
+{
+	return -1;	/* stub */
+}
+
+__be32
+nfsd4_sequence(struct svc_rqst *r,
+	       struct nfsd4_compound_state *cstate,
+	       struct nfsd4_sequence *seq)
+{
+	return -1;	/* stub */
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 __be32
 nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		  struct nfsd4_setclientid *setclid)
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 40b148e..0148d54 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -475,6 +475,20 @@ extern __be32 nfsd4_setclientid(struct svc_rqst *rqstp,
 extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 		struct nfsd4_setclientid_confirm *setclientid_confirm);
+#if defined(CONFIG_NFSD_V4_1)
+extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
+		struct nfsd4_compound_state *,
+struct nfsd4_exchange_id *);
+		extern __be32 nfsd4_create_session(struct svc_rqst *,
+		struct nfsd4_compound_state *,
+		struct nfsd4_create_session *);
+extern __be32 nfsd4_sequence(struct svc_rqst *,
+		struct nfsd4_compound_state *,
+		struct nfsd4_sequence *);
+extern __be32 nfsd4_destroy_session(struct svc_rqst *,
+		struct nfsd4_compound_state *,
+		struct nfsd4_destroy_session *);
+#endif /* CONFIG_NFSD_V4_1 */
 extern __be32 nfsd4_process_open1(struct nfsd4_open *open);
 extern __be32 nfsd4_process_open2(struct svc_rqst *rqstp,
 		struct svc_fh *current_fh, struct nfsd4_open *open);
-- 
1.6.2.1

[PATCH v2 15/47] nfsd41: exchange_id operation

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the exchange_id operation confoming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-28

Based on the client provided name, hash a client id.
If a confirmed one is found, compare the op's creds and
verifier.  If the creds match and the verifier is different
then expire the old client (client re-incarnated), otherwise,
if both match, assume it's a replay and ignore it.

If an unconfirmed client is found, then copy the new creds
and verifer if need update, otherwise assume replay.

The client is moved to a confirmed state on create_session.

In the nfs41 branch set the exchange_id flags to
EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_SUPP_MOVED_REFER
(pNFS is not supported, Referrals are supported,
Migration is not.).

Address various scenarios from section 18.35 of the spec:

1. Check for EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and set
   EXCHGID4_FLAG_CONFIRMED_R as appropriate.

2. Return error codes per 18.35.4 scenarios.

3. Update client records or generate new client ids depending on
   scenario.

Note: 18.35.4 case 3 probably still needs revisiting.  The handling
seems not quite right.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamosn <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use utsname for major_id (and copy to server_scope)]
[nfsd41: fix handling of various exchange id scenarios]
Signed-off-by: Mike Sager <sager@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |  138 +++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c          |  146 +++++++++++++++++++++++++++++++++++++++++++-
 include/linux/nfsd/state.h |    2 +
 include/linux/nfsd/xdr4.h  |    8 ++-
 4 files changed, 289 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index bbb7455..09c63ff 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -841,12 +841,148 @@ out_err:
 }
 
 #if defined(CONFIG_NFSD_V4_1)
+/*
+ * Set the exchange_id flags returned by the server.
+ */
+static void
+nfsd4_set_ex_flags(struct nfs4_client *new, struct nfsd4_exchange_id *clid)
+{
+	/* pNFS is not supported */
+	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_NON_PNFS;
+
+	/* Referrals are supported, Migration is not. */
+	new->cl_exchange_flags |= EXCHGID4_FLAG_SUPP_MOVED_REFER;
+
+	/* set the wire flags to return to client. */
+	clid->flags = new->cl_exchange_flags;
+}
+
 __be32
 nfsd4_exchange_id(struct svc_rqst *rqstp,
 		  struct nfsd4_compound_state *cstate,
 		  struct nfsd4_exchange_id *exid)
 {
-	return -1;	/* stub */
+	struct nfs4_client *unconf, *conf, *new;
+	int status;
+	unsigned int		strhashval;
+	char			dname[HEXDIR_LEN];
+	nfs4_verifier		verf = exid->verifier;
+	u32			ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
+	struct xdr_netobj clname = {
+		.len = exid->id_len,
+		.data = exid->id,
+	};
+
+	dprintk("%s rqstp=%p exid=%p clname.len=%u clname.data=%p "
+		" ip_addr=%u flags %x, spa_how %d\n",
+		__func__, rqstp, exid, clname.len, clname.data,
+		ip_addr, exid->flags, exid->spa_how);
+
+	if (!check_name(clname) || (exid->flags & EXCHGID4_INVAL_FLAG_MASK_A))
+		return nfserr_inval;
+
+	/* Currently only support SP4_NONE */
+	if (exid->spa_how != SP4_NONE)
+		return nfserr_encr_alg_unsupp;
+
+	status = nfs4_make_rec_clidname(dname, &clname);
+
+	if (status)
+		goto error;
+
+	strhashval = clientstr_hashval(dname);
+
+	nfs4_lock_state();
+	status = nfs_ok;
+
+	conf = find_confirmed_client_by_str(dname, strhashval);
+	if (conf) {
+		if (!same_verf(&verf, &conf->cl_verifier)) {
+			/* 18.35.4 case 8 */
+			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
+				status = nfserr_not_same;
+				goto out;
+			}
+			/* Client reboot: destroy old state */
+			expire_client(conf);
+			goto out_new;
+		}
+		if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) {
+			/* 18.35.4 case 9 */
+			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
+				status = nfserr_perm;
+				goto out;
+			}
+			expire_client(conf);
+			goto out_new;
+		}
+		if (ip_addr != conf->cl_addr &&
+		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A)) {
+			/* Client collision. 18.35.4 case 3 */
+			status = nfserr_clid_inuse;
+			goto out;
+		}
+		/*
+		 * Set bit when the owner id and verifier map to an already
+		 * confirmed client id (18.35.3).
+		 */
+		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
+
+		/*
+		 * Falling into 18.35.4 case 2, possible router replay.
+		 * Leave confirmed record intact and return same result.
+		 */
+		copy_verf(conf, &verf);
+		new = conf;
+		goto out_copy;
+	} else {
+		/* 18.35.4 case 7 */
+		if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
+			status = nfserr_noent;
+			goto out;
+		}
+	}
+
+	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
+	if (unconf) {
+		/*
+		 * Possible retry or client restart.  Per 18.35.4 case 4,
+		 * a new unconfirmed record should be generated regardless
+		 * of whether any properties have changed.
+		 */
+		expire_client(unconf);
+	}
+
+out_new:
+	/* Normal case */
+	new = create_client(clname, dname);
+	if (new == NULL) {
+		status = nfserr_resource;
+		goto out;
+	}
+
+	copy_verf(new, &verf);
+	copy_cred(&new->cl_cred, &rqstp->rq_cred);
+	new->cl_addr = ip_addr;
+	gen_clid(new);
+	gen_confirm(new);
+	add_to_unconfirmed(new, strhashval);
+out_copy:
+	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
+	exid->clientid.cl_id = new->cl_clientid.cl_id;
+
+	new->cl_seqid = exid->seqid = 1;
+	nfsd4_set_ex_flags(new, exid);
+
+	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
+		new->cl_seqid, new->cl_exchange_flags);
+	status = nfs_ok;
+
+out:
+	nfs4_unlock_state();
+error:
+	dprintk("nfsd4_exchange_id returns %d\n", ntohl(status));
+	return status;
 }
 
 __be32
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index b082d07..840cf6a 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -45,6 +45,7 @@
 #include <linux/fs.h>
 #include <linux/namei.h>
 #include <linux/vfs.h>
+#include <linux/utsname.h>
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
@@ -999,9 +1000,100 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
 #if defined(CONFIG_NFSD_V4_1)
 static __be32
 nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
-			 struct nfsd4_exchange_id *clid)
+			 struct nfsd4_exchange_id *exid)
 {
-	return nfserr_opnotsupp;	/* stub */
+	int dummy;
+	DECODE_HEAD;
+
+	READ_BUF(NFS4_VERIFIER_SIZE);
+	COPYMEM(exid->verifier.data, NFS4_VERIFIER_SIZE);
+
+	READ_BUF(4);
+	READ32(exid->id_len);
+
+	READ_BUF(exid->id_len);
+	SAVEMEM(exid->id, exid->id_len);
+
+	READ_BUF(4);
+	READ32(exid->flags);
+
+	/* Ignore state_protect4_a */
+	READ_BUF(4);
+	READ32(exid->spa_how);
+	switch (exid->spa_how) {
+	case SP4_NONE:
+		break;
+	case SP4_MACH_CRED:
+		/* spo_must_enforce */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+
+		/* spo_must_allow */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+		break;
+	case SP4_SSV:
+		/* ssp_ops */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy * 4);
+		p += dummy;
+
+		/* ssp_hash_algs<> */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* ssp_encr_algs<> */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* ssp_window and ssp_num_gss_handles */
+		READ_BUF(8);
+		READ32(dummy);
+		READ32(dummy);
+		break;
+	default:
+		goto xdr_error;
+	}
+
+	/* Ignore Implementation ID */
+	READ_BUF(4);    /* nfs_impl_id4 array length */
+	READ32(dummy);
+
+	if (dummy > 1)
+		goto xdr_error;
+
+	if (dummy == 1) {
+		/* nii_domain */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* nii_name */
+		READ_BUF(4);
+		READ32(dummy);
+		READ_BUF(dummy);
+		p += XDR_QUADLEN(dummy);
+
+		/* nii_date */
+		READ_BUF(12);
+		p += 3;
+	}
+	DECODE_TAIL;
 }
 
 static __be32
@@ -2672,7 +2764,55 @@ static __be32
 nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
 			 struct nfsd4_exchange_id *exid)
 {
-	/* stub */
+	ENCODE_HEAD;
+	char *major_id;
+	char *server_scope;
+	int major_id_sz;
+	int server_scope_sz;
+	uint64_t minor_id = 0;
+
+	if (nfserr)
+		goto out;
+
+	major_id = utsname()->nodename;
+	major_id_sz = strlen(major_id);
+	server_scope = utsname()->nodename;
+	server_scope_sz = strlen(server_scope);
+
+	RESERVE_SPACE(
+		8 /* eir_clientid */ +
+		4 /* eir_sequenceid */ +
+		4 /* eir_flags */ +
+		4 /* spr_how (SP4_NONE) */ +
+		8 /* so_minor_id */ +
+		4 /* so_major_id.len */ +
+		(XDR_QUADLEN(major_id_sz) * 4) +
+		4 /* eir_server_scope.len */ +
+		(XDR_QUADLEN(server_scope_sz) * 4) +
+		4 /* eir_server_impl_id.count (0) */);
+
+	WRITEMEM(&exid->clientid, 8);
+	WRITE32(exid->seqid);
+	WRITE32(exid->flags);
+
+	/* state_protect4_r. Currently only support SP4_NONE */
+	BUG_ON(exid->spa_how != SP4_NONE);
+	WRITE32(exid->spa_how);
+
+	/* The server_owner struct */
+	WRITE64(minor_id);      /* Minor id */
+	/* major id */
+	WRITE32(major_id_sz);
+	WRITEMEM(major_id, major_id_sz);
+
+	/* Server scope */
+	WRITE32(server_scope_sz);
+	WRITEMEM(server_scope, server_scope_sz);
+
+	/* Implementation id */
+	WRITE32(0);	/* zero length nfs_impl_id4 array */
+	ADJUST_ARGS();
+out:
 	return nfserr;
 }
 
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 7592d7b..5de36a7 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -173,6 +173,8 @@ struct nfs4_client {
 	u32			cl_firststate;	/* recovery dir creation */
 #ifdef CONFIG_NFSD_V4_1
 	struct list_head	cl_sessions;
+	u32			cl_seqid;       /* seqid for create_session */
+	u32			cl_exchange_flags;
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 0148d54..ea5a427 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -348,7 +348,13 @@ struct nfsd4_write {
 
 #if defined(CONFIG_NFSD_V4_1)
 struct nfsd4_exchange_id {
-	int	foo;	/* stub */
+	nfs4_verifier	verifier;
+	u32		id_len;
+	char		*id;
+	u32		flags;
+	clientid_t	clientid;
+	u32		seqid;
+	int		spa_how;
 };
 
 struct nfsd4_create_session {
-- 
1.6.2.1

Re: [PATCH v2 15/47] nfsd41: exchange_id operation

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:12AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Implement the exchange_id operation confoming to
> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-28
> 
> Based on the client provided name, hash a client id.
> If a confirmed one is found, compare the op's creds and
> verifier.  If the creds match and the verifier is different
> then expire the old client (client re-incarnated), otherwise,
> if both match, assume it's a replay and ignore it.
> 
> If an unconfirmed client is found, then copy the new creds
> and verifer if need update, otherwise assume replay.
> 
> The client is moved to a confirmed state on create_session.
> 
> In the nfs41 branch set the exchange_id flags to
> EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_SUPP_MOVED_REFER
> (pNFS is not supported, Referrals are supported,
> Migration is not.).
> 
> Address various scenarios from section 18.35 of the spec:
> 
> 1. Check for EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and set
>    EXCHGID4_FLAG_CONFIRMED_R as appropriate.
> 
> 2. Return error codes per 18.35.4 scenarios.
> 
> 3. Update client records or generate new client ids depending on
>    scenario.
> 
> Note: 18.35.4 case 3 probably still needs revisiting.  The handling
> seems not quite right.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> Signed-off-by: Andy Adamosn <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: use utsname for major_id (and copy to server_scope)]
> [nfsd41: fix handling of various exchange id scenarios]
> Signed-off-by: Mike Sager <sager@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |  138 +++++++++++++++++++++++++++++++++++++++++-
>  fs/nfsd/nfs4xdr.c          |  146 +++++++++++++++++++++++++++++++++++++++++++-
>  include/linux/nfsd/state.h |    2 +
>  include/linux/nfsd/xdr4.h  |    8 ++-
>  4 files changed, 289 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index bbb7455..09c63ff 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -841,12 +841,148 @@ out_err:
>  }
>  
>  #if defined(CONFIG_NFSD_V4_1)
> +/*
> + * Set the exchange_id flags returned by the server.
> + */
> +static void
> +nfsd4_set_ex_flags(struct nfs4_client *new, struct nfsd4_exchange_id *clid)
> +{
> +	/* pNFS is not supported */
> +	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_NON_PNFS;
> +
> +	/* Referrals are supported, Migration is not. */
> +	new->cl_exchange_flags |= EXCHGID4_FLAG_SUPP_MOVED_REFER;
> +
> +	/* set the wire flags to return to client. */
> +	clid->flags = new->cl_exchange_flags;

Hm.  At this point we could do away with cl_exchange_flags and just
unconditionally return the above two bits.

I guess this will change with pNFS?  OK.

> +}
> +
>  __be32
>  nfsd4_exchange_id(struct svc_rqst *rqstp,
>  		  struct nfsd4_compound_state *cstate,
>  		  struct nfsd4_exchange_id *exid)
>  {
> -	return -1;	/* stub */
> +	struct nfs4_client *unconf, *conf, *new;
> +	int status;
> +	unsigned int		strhashval;
> +	char			dname[HEXDIR_LEN];
> +	nfs4_verifier		verf = exid->verifier;
> +	u32			ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
> +	struct xdr_netobj clname = {
> +		.len = exid->id_len,
> +		.data = exid->id,
> +	};

Would it simplify things just to embed an xdr_netobj in
nfsd4_exchange_id?

> +
> +	dprintk("%s rqstp=%p exid=%p clname.len=%u clname.data=%p "
> +		" ip_addr=%u flags %x, spa_how %d\n",
> +		__func__, rqstp, exid, clname.len, clname.data,
> +		ip_addr, exid->flags, exid->spa_how);
> +
> +	if (!check_name(clname) || (exid->flags & EXCHGID4_INVAL_FLAG_MASK_A))
> +		return nfserr_inval;
> +
> +	/* Currently only support SP4_NONE */
> +	if (exid->spa_how != SP4_NONE)
> +		return nfserr_encr_alg_unsupp;

Isn't support for the others mandatory?  Let's just make this
serverfault, in that case--this is a bug in the server.  It'll be a
reminder that we need to fix this....

> +
> +	status = nfs4_make_rec_clidname(dname, &clname);
> +
> +	if (status)
> +		goto error;
> +
> +	strhashval = clientstr_hashval(dname);
> +
> +	nfs4_lock_state();
> +	status = nfs_ok;
> +
> +	conf = find_confirmed_client_by_str(dname, strhashval);
> +	if (conf) {
> +		if (!same_verf(&verf, &conf->cl_verifier)) {
> +			/* 18.35.4 case 8 */
> +			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
> +				status = nfserr_not_same;
> +				goto out;
> +			}
> +			/* Client reboot: destroy old state */
> +			expire_client(conf);
> +			goto out_new;
> +		}
> +		if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) {
> +			/* 18.35.4 case 9 */
> +			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
> +				status = nfserr_perm;
> +				goto out;
> +			}
> +			expire_client(conf);
> +			goto out_new;
> +		}
> +		if (ip_addr != conf->cl_addr &&

Why the ip_addr comparison?

--b.

> +		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A)) {
> +			/* Client collision. 18.35.4 case 3 */
> +			status = nfserr_clid_inuse;
> +			goto out;
> +		}
> +		/*
> +		 * Set bit when the owner id and verifier map to an already
> +		 * confirmed client id (18.35.3).
> +		 */
> +		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
> +
> +		/*
> +		 * Falling into 18.35.4 case 2, possible router replay.
> +		 * Leave confirmed record intact and return same result.
> +		 */
> +		copy_verf(conf, &verf);
> +		new = conf;
> +		goto out_copy;
> +	} else {
> +		/* 18.35.4 case 7 */
> +		if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
> +			status = nfserr_noent;
> +			goto out;
> +		}
> +	}
> +
> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
> +	if (unconf) {
> +		/*
> +		 * Possible retry or client restart.  Per 18.35.4 case 4,
> +		 * a new unconfirmed record should be generated regardless
> +		 * of whether any properties have changed.
> +		 */
> +		expire_client(unconf);
> +	}
> +
> +out_new:
> +	/* Normal case */
> +	new = create_client(clname, dname);
> +	if (new == NULL) {
> +		status = nfserr_resource;
> +		goto out;
> +	}
> +
> +	copy_verf(new, &verf);
> +	copy_cred(&new->cl_cred, &rqstp->rq_cred);
> +	new->cl_addr = ip_addr;
> +	gen_clid(new);
> +	gen_confirm(new);
> +	add_to_unconfirmed(new, strhashval);
> +out_copy:
> +	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
> +	exid->clientid.cl_id = new->cl_clientid.cl_id;
> +
> +	new->cl_seqid = exid->seqid = 1;
> +	nfsd4_set_ex_flags(new, exid);
> +
> +	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
> +		new->cl_seqid, new->cl_exchange_flags);
> +	status = nfs_ok;
> +
> +out:
> +	nfs4_unlock_state();
> +error:
> +	dprintk("nfsd4_exchange_id returns %d\n", ntohl(status));
> +	return status;
>  }
>  
>  __be32
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index b082d07..840cf6a 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -45,6 +45,7 @@
>  #include <linux/fs.h>
>  #include <linux/namei.h>
>  #include <linux/vfs.h>
> +#include <linux/utsname.h>
>  #include <linux/sunrpc/xdr.h>
>  #include <linux/sunrpc/svc.h>
>  #include <linux/sunrpc/clnt.h>
> @@ -999,9 +1000,100 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
>  #if defined(CONFIG_NFSD_V4_1)
>  static __be32
>  nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
> -			 struct nfsd4_exchange_id *clid)
> +			 struct nfsd4_exchange_id *exid)
>  {
> -	return nfserr_opnotsupp;	/* stub */
> +	int dummy;
> +	DECODE_HEAD;
> +
> +	READ_BUF(NFS4_VERIFIER_SIZE);
> +	COPYMEM(exid->verifier.data, NFS4_VERIFIER_SIZE);
> +
> +	READ_BUF(4);
> +	READ32(exid->id_len);
> +
> +	READ_BUF(exid->id_len);
> +	SAVEMEM(exid->id, exid->id_len);
> +
> +	READ_BUF(4);
> +	READ32(exid->flags);
> +
> +	/* Ignore state_protect4_a */
> +	READ_BUF(4);
> +	READ32(exid->spa_how);
> +	switch (exid->spa_how) {
> +	case SP4_NONE:
> +		break;
> +	case SP4_MACH_CRED:
> +		/* spo_must_enforce */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +
> +		/* spo_must_allow */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +		break;
> +	case SP4_SSV:
> +		/* ssp_ops */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +
> +		/* ssp_hash_algs<> */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* ssp_encr_algs<> */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* ssp_window and ssp_num_gss_handles */
> +		READ_BUF(8);
> +		READ32(dummy);
> +		READ32(dummy);
> +		break;
> +	default:
> +		goto xdr_error;
> +	}
> +
> +	/* Ignore Implementation ID */
> +	READ_BUF(4);    /* nfs_impl_id4 array length */
> +	READ32(dummy);
> +
> +	if (dummy > 1)
> +		goto xdr_error;
> +
> +	if (dummy == 1) {
> +		/* nii_domain */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* nii_name */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* nii_date */
> +		READ_BUF(12);
> +		p += 3;
> +	}
> +	DECODE_TAIL;
>  }
>  
>  static __be32
> @@ -2672,7 +2764,55 @@ static __be32
>  nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
>  			 struct nfsd4_exchange_id *exid)
>  {
> -	/* stub */
> +	ENCODE_HEAD;
> +	char *major_id;
> +	char *server_scope;
> +	int major_id_sz;
> +	int server_scope_sz;
> +	uint64_t minor_id = 0;
> +
> +	if (nfserr)
> +		goto out;
> +
> +	major_id = utsname()->nodename;
> +	major_id_sz = strlen(major_id);
> +	server_scope = utsname()->nodename;
> +	server_scope_sz = strlen(server_scope);
> +
> +	RESERVE_SPACE(
> +		8 /* eir_clientid */ +
> +		4 /* eir_sequenceid */ +
> +		4 /* eir_flags */ +
> +		4 /* spr_how (SP4_NONE) */ +
> +		8 /* so_minor_id */ +
> +		4 /* so_major_id.len */ +
> +		(XDR_QUADLEN(major_id_sz) * 4) +
> +		4 /* eir_server_scope.len */ +
> +		(XDR_QUADLEN(server_scope_sz) * 4) +
> +		4 /* eir_server_impl_id.count (0) */);
> +
> +	WRITEMEM(&exid->clientid, 8);
> +	WRITE32(exid->seqid);
> +	WRITE32(exid->flags);
> +
> +	/* state_protect4_r. Currently only support SP4_NONE */
> +	BUG_ON(exid->spa_how != SP4_NONE);
> +	WRITE32(exid->spa_how);
> +
> +	/* The server_owner struct */
> +	WRITE64(minor_id);      /* Minor id */
> +	/* major id */
> +	WRITE32(major_id_sz);
> +	WRITEMEM(major_id, major_id_sz);
> +
> +	/* Server scope */
> +	WRITE32(server_scope_sz);
> +	WRITEMEM(server_scope, server_scope_sz);
> +
> +	/* Implementation id */
> +	WRITE32(0);	/* zero length nfs_impl_id4 array */
> +	ADJUST_ARGS();
> +out:
>  	return nfserr;
>  }
>  
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 7592d7b..5de36a7 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -173,6 +173,8 @@ struct nfs4_client {
>  	u32			cl_firststate;	/* recovery dir creation */
>  #ifdef CONFIG_NFSD_V4_1
>  	struct list_head	cl_sessions;
> +	u32			cl_seqid;       /* seqid for create_session */
> +	u32			cl_exchange_flags;
>  #endif /* CONFIG_NFSD_V4_1 */
>  };
>  
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 0148d54..ea5a427 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -348,7 +348,13 @@ struct nfsd4_write {
>  
>  #if defined(CONFIG_NFSD_V4_1)
>  struct nfsd4_exchange_id {
> -	int	foo;	/* stub */
> +	nfs4_verifier	verifier;
> +	u32		id_len;
> +	char		*id;
> +	u32		flags;
> +	clientid_t	clientid;
> +	u32		seqid;
> +	int		spa_how;
>  };
>  
>  struct nfsd4_create_session {
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 15/47] nfsd41: exchange_id operation

[Benny Halevy]

On Mar. 31, 2009, 1:06 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:32:12AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Implement the exchange_id operation confoming to
>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-28
>>
>> Based on the client provided name, hash a client id.
>> If a confirmed one is found, compare the op's creds and
>> verifier.  If the creds match and the verifier is different
>> then expire the old client (client re-incarnated), otherwise,
>> if both match, assume it's a replay and ignore it.
>>
>> If an unconfirmed client is found, then copy the new creds
>> and verifer if need update, otherwise assume replay.
>>
>> The client is moved to a confirmed state on create_session.
>>
>> In the nfs41 branch set the exchange_id flags to
>> EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_SUPP_MOVED_REFER
>> (pNFS is not supported, Referrals are supported,
>> Migration is not.).
>>
>> Address various scenarios from section 18.35 of the spec:
>>
>> 1. Check for EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and set
>>    EXCHGID4_FLAG_CONFIRMED_R as appropriate.
>>
>> 2. Return error codes per 18.35.4 scenarios.
>>
>> 3. Update client records or generate new client ids depending on
>>    scenario.
>>
>> Note: 18.35.4 case 3 probably still needs revisiting.  The handling
>> seems not quite right.
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> Signed-off-by: Andy Adamosn <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfsd41: use utsname for major_id (and copy to server_scope)]
>> [nfsd41: fix handling of various exchange id scenarios]
>> Signed-off-by: Mike Sager <sager@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4state.c        |  138 +++++++++++++++++++++++++++++++++++++++++-
>>  fs/nfsd/nfs4xdr.c          |  146 +++++++++++++++++++++++++++++++++++++++++++-
>>  include/linux/nfsd/state.h |    2 +
>>  include/linux/nfsd/xdr4.h  |    8 ++-
>>  4 files changed, 289 insertions(+), 5 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index bbb7455..09c63ff 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -841,12 +841,148 @@ out_err:
>>  }
>>  
>>  #if defined(CONFIG_NFSD_V4_1)
>> +/*
>> + * Set the exchange_id flags returned by the server.
>> + */
>> +static void
>> +nfsd4_set_ex_flags(struct nfs4_client *new, struct nfsd4_exchange_id *clid)
>> +{
>> +	/* pNFS is not supported */
>> +	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_NON_PNFS;
>> +
>> +	/* Referrals are supported, Migration is not. */
>> +	new->cl_exchange_flags |= EXCHGID4_FLAG_SUPP_MOVED_REFER;
>> +
>> +	/* set the wire flags to return to client. */
>> +	clid->flags = new->cl_exchange_flags;
> 
> Hm.  At this point we could do away with cl_exchange_flags and just
> unconditionally return the above two bits.
> 
> I guess this will change with pNFS?  OK.

True.  Also, we also use keep cl_exchange_flags for
differentiating between 4.0 and 4.1 clientids.
(see "[PATCH v2 16/47] nfsd41: match clientid establishment method")

> 
>> +}
>> +
>>  __be32
>>  nfsd4_exchange_id(struct svc_rqst *rqstp,
>>  		  struct nfsd4_compound_state *cstate,
>>  		  struct nfsd4_exchange_id *exid)
>>  {
>> -	return -1;	/* stub */
>> +	struct nfs4_client *unconf, *conf, *new;
>> +	int status;
>> +	unsigned int		strhashval;
>> +	char			dname[HEXDIR_LEN];
>> +	nfs4_verifier		verf = exid->verifier;
>> +	u32			ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
>> +	struct xdr_netobj clname = {
>> +		.len = exid->id_len,
>> +		.data = exid->id,
>> +	};
> 
> Would it simplify things just to embed an xdr_netobj in
> nfsd4_exchange_id?

Yeah, looks good to me.

> 
>> +
>> +	dprintk("%s rqstp=%p exid=%p clname.len=%u clname.data=%p "
>> +		" ip_addr=%u flags %x, spa_how %d\n",
>> +		__func__, rqstp, exid, clname.len, clname.data,
>> +		ip_addr, exid->flags, exid->spa_how);
>> +
>> +	if (!check_name(clname) || (exid->flags & EXCHGID4_INVAL_FLAG_MASK_A))
>> +		return nfserr_inval;
>> +
>> +	/* Currently only support SP4_NONE */
>> +	if (exid->spa_how != SP4_NONE)
>> +		return nfserr_encr_alg_unsupp;
> 
> Isn't support for the others mandatory?  Let's just make this
> serverfault, in that case--this is a bug in the server.  It'll be a
> reminder that we need to fix this....

True. nfserr_encr_alg_unsupp is valid only for ssp_encr_algs.
Andy, I believe you're the author of this. OK with you to return
nfserr_serverfault instead?

> 
>> +
>> +	status = nfs4_make_rec_clidname(dname, &clname);
>> +
>> +	if (status)
>> +		goto error;
>> +
>> +	strhashval = clientstr_hashval(dname);
>> +
>> +	nfs4_lock_state();
>> +	status = nfs_ok;
>> +
>> +	conf = find_confirmed_client_by_str(dname, strhashval);
>> +	if (conf) {
>> +		if (!same_verf(&verf, &conf->cl_verifier)) {
>> +			/* 18.35.4 case 8 */
>> +			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
>> +				status = nfserr_not_same;
>> +				goto out;
>> +			}
>> +			/* Client reboot: destroy old state */
>> +			expire_client(conf);
>> +			goto out_new;
>> +		}
>> +		if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) {
>> +			/* 18.35.4 case 9 */
>> +			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
>> +				status = nfserr_perm;
>> +				goto out;
>> +			}
>> +			expire_client(conf);
>> +			goto out_new;
>> +		}
>> +		if (ip_addr != conf->cl_addr &&
> 
> Why the ip_addr comparison?

Good question.
IIRC this covers the client restart case (18.35.4 case 5).
I.e., we got an EXCHANGE_ID updating a confirmed clientid.
We got this far, meaning it has same ownerid, verifier, and creds
and EXCHGID4_FLAG_UPD_CONFIRMED_REC_A is not set (or actually
we don't care as we update the confirmed client either via
case 3 or case 5 of the spec.).

Still, in case 5, client restart, we can't the client come up
with a new IP address (say due to, e.g., DHCP :)

Benny

> 
> --b.
> 
>> +		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A)) {
>> +			/* Client collision. 18.35.4 case 3 */
>> +			status = nfserr_clid_inuse;
>> +			goto out;
>> +		}
>> +		/*
>> +		 * Set bit when the owner id and verifier map to an already
>> +		 * confirmed client id (18.35.3).
>> +		 */
>> +		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
>> +
>> +		/*
>> +		 * Falling into 18.35.4 case 2, possible router replay.
>> +		 * Leave confirmed record intact and return same result.
>> +		 */
>> +		copy_verf(conf, &verf);
>> +		new = conf;
>> +		goto out_copy;
>> +	} else {
>> +		/* 18.35.4 case 7 */
>> +		if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
>> +			status = nfserr_noent;
>> +			goto out;
>> +		}
>> +	}
>> +
>> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
>> +	if (unconf) {
>> +		/*
>> +		 * Possible retry or client restart.  Per 18.35.4 case 4,
>> +		 * a new unconfirmed record should be generated regardless
>> +		 * of whether any properties have changed.
>> +		 */
>> +		expire_client(unconf);
>> +	}
>> +
>> +out_new:
>> +	/* Normal case */
>> +	new = create_client(clname, dname);
>> +	if (new == NULL) {
>> +		status = nfserr_resource;
>> +		goto out;
>> +	}
>> +
>> +	copy_verf(new, &verf);
>> +	copy_cred(&new->cl_cred, &rqstp->rq_cred);
>> +	new->cl_addr = ip_addr;
>> +	gen_clid(new);
>> +	gen_confirm(new);
>> +	add_to_unconfirmed(new, strhashval);
>> +out_copy:
>> +	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
>> +	exid->clientid.cl_id = new->cl_clientid.cl_id;
>> +
>> +	new->cl_seqid = exid->seqid = 1;
>> +	nfsd4_set_ex_flags(new, exid);
>> +
>> +	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
>> +		new->cl_seqid, new->cl_exchange_flags);
>> +	status = nfs_ok;
>> +
>> +out:
>> +	nfs4_unlock_state();
>> +error:
>> +	dprintk("nfsd4_exchange_id returns %d\n", ntohl(status));
>> +	return status;
>>  }
>>  
>>  __be32
>> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
>> index b082d07..840cf6a 100644
>> --- a/fs/nfsd/nfs4xdr.c
>> +++ b/fs/nfsd/nfs4xdr.c
>> @@ -45,6 +45,7 @@
>>  #include <linux/fs.h>
>>  #include <linux/namei.h>
>>  #include <linux/vfs.h>
>> +#include <linux/utsname.h>
>>  #include <linux/sunrpc/xdr.h>
>>  #include <linux/sunrpc/svc.h>
>>  #include <linux/sunrpc/clnt.h>
>> @@ -999,9 +1000,100 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
>>  #if defined(CONFIG_NFSD_V4_1)
>>  static __be32
>>  nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
>> -			 struct nfsd4_exchange_id *clid)
>> +			 struct nfsd4_exchange_id *exid)
>>  {
>> -	return nfserr_opnotsupp;	/* stub */
>> +	int dummy;
>> +	DECODE_HEAD;
>> +
>> +	READ_BUF(NFS4_VERIFIER_SIZE);
>> +	COPYMEM(exid->verifier.data, NFS4_VERIFIER_SIZE);
>> +
>> +	READ_BUF(4);
>> +	READ32(exid->id_len);
>> +
>> +	READ_BUF(exid->id_len);
>> +	SAVEMEM(exid->id, exid->id_len);
>> +
>> +	READ_BUF(4);
>> +	READ32(exid->flags);
>> +
>> +	/* Ignore state_protect4_a */
>> +	READ_BUF(4);
>> +	READ32(exid->spa_how);
>> +	switch (exid->spa_how) {
>> +	case SP4_NONE:
>> +		break;
>> +	case SP4_MACH_CRED:
>> +		/* spo_must_enforce */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy * 4);
>> +		p += dummy;
>> +
>> +		/* spo_must_allow */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy * 4);
>> +		p += dummy;
>> +		break;
>> +	case SP4_SSV:
>> +		/* ssp_ops */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy * 4);
>> +		p += dummy;
>> +
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy * 4);
>> +		p += dummy;
>> +
>> +		/* ssp_hash_algs<> */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy);
>> +		p += XDR_QUADLEN(dummy);
>> +
>> +		/* ssp_encr_algs<> */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy);
>> +		p += XDR_QUADLEN(dummy);
>> +
>> +		/* ssp_window and ssp_num_gss_handles */
>> +		READ_BUF(8);
>> +		READ32(dummy);
>> +		READ32(dummy);
>> +		break;
>> +	default:
>> +		goto xdr_error;
>> +	}
>> +
>> +	/* Ignore Implementation ID */
>> +	READ_BUF(4);    /* nfs_impl_id4 array length */
>> +	READ32(dummy);
>> +
>> +	if (dummy > 1)
>> +		goto xdr_error;
>> +
>> +	if (dummy == 1) {
>> +		/* nii_domain */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy);
>> +		p += XDR_QUADLEN(dummy);
>> +
>> +		/* nii_name */
>> +		READ_BUF(4);
>> +		READ32(dummy);
>> +		READ_BUF(dummy);
>> +		p += XDR_QUADLEN(dummy);
>> +
>> +		/* nii_date */
>> +		READ_BUF(12);
>> +		p += 3;
>> +	}
>> +	DECODE_TAIL;
>>  }
>>  
>>  static __be32
>> @@ -2672,7 +2764,55 @@ static __be32
>>  nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
>>  			 struct nfsd4_exchange_id *exid)
>>  {
>> -	/* stub */
>> +	ENCODE_HEAD;
>> +	char *major_id;
>> +	char *server_scope;
>> +	int major_id_sz;
>> +	int server_scope_sz;
>> +	uint64_t minor_id = 0;
>> +
>> +	if (nfserr)
>> +		goto out;
>> +
>> +	major_id = utsname()->nodename;
>> +	major_id_sz = strlen(major_id);
>> +	server_scope = utsname()->nodename;
>> +	server_scope_sz = strlen(server_scope);
>> +
>> +	RESERVE_SPACE(
>> +		8 /* eir_clientid */ +
>> +		4 /* eir_sequenceid */ +
>> +		4 /* eir_flags */ +
>> +		4 /* spr_how (SP4_NONE) */ +
>> +		8 /* so_minor_id */ +
>> +		4 /* so_major_id.len */ +
>> +		(XDR_QUADLEN(major_id_sz) * 4) +
>> +		4 /* eir_server_scope.len */ +
>> +		(XDR_QUADLEN(server_scope_sz) * 4) +
>> +		4 /* eir_server_impl_id.count (0) */);
>> +
>> +	WRITEMEM(&exid->clientid, 8);
>> +	WRITE32(exid->seqid);
>> +	WRITE32(exid->flags);
>> +
>> +	/* state_protect4_r. Currently only support SP4_NONE */
>> +	BUG_ON(exid->spa_how != SP4_NONE);
>> +	WRITE32(exid->spa_how);
>> +
>> +	/* The server_owner struct */
>> +	WRITE64(minor_id);      /* Minor id */
>> +	/* major id */
>> +	WRITE32(major_id_sz);
>> +	WRITEMEM(major_id, major_id_sz);
>> +
>> +	/* Server scope */
>> +	WRITE32(server_scope_sz);
>> +	WRITEMEM(server_scope, server_scope_sz);
>> +
>> +	/* Implementation id */
>> +	WRITE32(0);	/* zero length nfs_impl_id4 array */
>> +	ADJUST_ARGS();
>> +out:
>>  	return nfserr;
>>  }
>>  
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index 7592d7b..5de36a7 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -173,6 +173,8 @@ struct nfs4_client {
>>  	u32			cl_firststate;	/* recovery dir creation */
>>  #ifdef CONFIG_NFSD_V4_1
>>  	struct list_head	cl_sessions;
>> +	u32			cl_seqid;       /* seqid for create_session */
>> +	u32			cl_exchange_flags;
>>  #endif /* CONFIG_NFSD_V4_1 */
>>  };
>>  
>> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
>> index 0148d54..ea5a427 100644
>> --- a/include/linux/nfsd/xdr4.h
>> +++ b/include/linux/nfsd/xdr4.h
>> @@ -348,7 +348,13 @@ struct nfsd4_write {
>>  
>>  #if defined(CONFIG_NFSD_V4_1)
>>  struct nfsd4_exchange_id {
>> -	int	foo;	/* stub */
>> +	nfs4_verifier	verifier;
>> +	u32		id_len;
>> +	char		*id;
>> +	u32		flags;
>> +	clientid_t	clientid;
>> +	u32		seqid;
>> +	int		spa_how;
>>  };
>>  
>>  struct nfsd4_create_session {
>> -- 
>> 1.6.2.1
>>


-- 
Benny Halevy
Software Architect
Panasas, Inc.
bhalevy@panasas.com
Tel/Fax: +972-3-647-8340
Mobile: +972-54-802-8340

Panasas: The Leader in Parallel Storage
www.panasas.com

Re: [PATCH v2 15/47] nfsd41: exchange_id operation

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:12AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Implement the exchange_id operation confoming to
> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-28
> 
> Based on the client provided name, hash a client id.
> If a confirmed one is found, compare the op's creds and
> verifier.  If the creds match and the verifier is different
> then expire the old client (client re-incarnated), otherwise,
> if both match, assume it's a replay and ignore it.
> 
> If an unconfirmed client is found, then copy the new creds
> and verifer if need update, otherwise assume replay.
> 
> The client is moved to a confirmed state on create_session.
> 
> In the nfs41 branch set the exchange_id flags to
> EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_SUPP_MOVED_REFER
> (pNFS is not supported, Referrals are supported,
> Migration is not.).
> 
> Address various scenarios from section 18.35 of the spec:
> 
> 1. Check for EXCHGID4_FLAG_UPD_CONFIRMED_REC_A and set
>    EXCHGID4_FLAG_CONFIRMED_R as appropriate.
> 
> 2. Return error codes per 18.35.4 scenarios.
> 
> 3. Update client records or generate new client ids depending on
>    scenario.
> 
> Note: 18.35.4 case 3 probably still needs revisiting.  The handling
> seems not quite right.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> Signed-off-by: Andy Adamosn <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: use utsname for major_id (and copy to server_scope)]
> [nfsd41: fix handling of various exchange id scenarios]
> Signed-off-by: Mike Sager <sager@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |  138 +++++++++++++++++++++++++++++++++++++++++-
>  fs/nfsd/nfs4xdr.c          |  146 +++++++++++++++++++++++++++++++++++++++++++-
>  include/linux/nfsd/state.h |    2 +
>  include/linux/nfsd/xdr4.h  |    8 ++-
>  4 files changed, 289 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index bbb7455..09c63ff 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -841,12 +841,148 @@ out_err:
>  }
>  
>  #if defined(CONFIG_NFSD_V4_1)
> +/*
> + * Set the exchange_id flags returned by the server.
> + */
> +static void
> +nfsd4_set_ex_flags(struct nfs4_client *new, struct nfsd4_exchange_id *clid)
> +{
> +	/* pNFS is not supported */
> +	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_NON_PNFS;
> +
> +	/* Referrals are supported, Migration is not. */
> +	new->cl_exchange_flags |= EXCHGID4_FLAG_SUPP_MOVED_REFER;
> +
> +	/* set the wire flags to return to client. */
> +	clid->flags = new->cl_exchange_flags;
> +}
> +
>  __be32
>  nfsd4_exchange_id(struct svc_rqst *rqstp,
>  		  struct nfsd4_compound_state *cstate,
>  		  struct nfsd4_exchange_id *exid)
>  {
> -	return -1;	/* stub */
> +	struct nfs4_client *unconf, *conf, *new;
> +	int status;
> +	unsigned int		strhashval;
> +	char			dname[HEXDIR_LEN];
> +	nfs4_verifier		verf = exid->verifier;
> +	u32			ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
> +	struct xdr_netobj clname = {
> +		.len = exid->id_len,
> +		.data = exid->id,
> +	};
> +
> +	dprintk("%s rqstp=%p exid=%p clname.len=%u clname.data=%p "
> +		" ip_addr=%u flags %x, spa_how %d\n",
> +		__func__, rqstp, exid, clname.len, clname.data,
> +		ip_addr, exid->flags, exid->spa_how);
> +
> +	if (!check_name(clname) || (exid->flags & EXCHGID4_INVAL_FLAG_MASK_A))
> +		return nfserr_inval;
> +
> +	/* Currently only support SP4_NONE */
> +	if (exid->spa_how != SP4_NONE)
> +		return nfserr_encr_alg_unsupp;
> +
> +	status = nfs4_make_rec_clidname(dname, &clname);
> +
> +	if (status)
> +		goto error;
> +
> +	strhashval = clientstr_hashval(dname);
> +
> +	nfs4_lock_state();
> +	status = nfs_ok;
> +
> +	conf = find_confirmed_client_by_str(dname, strhashval);
> +	if (conf) {
> +		if (!same_verf(&verf, &conf->cl_verifier)) {
> +			/* 18.35.4 case 8 */
> +			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
> +				status = nfserr_not_same;
> +				goto out;
> +			}
> +			/* Client reboot: destroy old state */
> +			expire_client(conf);

Surely you must need to check the creds before destroying the old state?

> +			goto out_new;
> +		}
> +		if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) {
> +			/* 18.35.4 case 9 */
> +			if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
> +				status = nfserr_perm;
> +				goto out;
> +			}
> +			expire_client(conf);

This expire_client() doesn't look right to me either.

> +			goto out_new;
> +		}
> +		if (ip_addr != conf->cl_addr &&
> +		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A)) {
> +			/* Client collision. 18.35.4 case 3 */
> +			status = nfserr_clid_inuse;
> +			goto out;
> +		}
> +		/*
> +		 * Set bit when the owner id and verifier map to an already
> +		 * confirmed client id (18.35.3).
> +		 */
> +		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
> +
> +		/*
> +		 * Falling into 18.35.4 case 2, possible router replay.

Checking the spec: case 2 says: "If the server has the following
confirmed record, and the request does not have
EXCHGID4_FLAG_UPD_CONFIRMED_REC_A set,..."

But that flag *is* set when we get to this code.

Isn't this case 6?

Could someone check these cases again very carefully?

> +		 * Leave confirmed record intact and return same result.
> +		 */
> +		copy_verf(conf, &verf);
> +		new = conf;
> +		goto out_copy;
> +	} else {

Note the "else" is redundant since all previous cases exit.

--b.

> +		/* 18.35.4 case 7 */
> +		if (exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A) {
> +			status = nfserr_noent;
> +			goto out;
> +		}
> +	}
> +
> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
> +	if (unconf) {
> +		/*
> +		 * Possible retry or client restart.  Per 18.35.4 case 4,
> +		 * a new unconfirmed record should be generated regardless
> +		 * of whether any properties have changed.
> +		 */
> +		expire_client(unconf);
> +	}
> +
> +out_new:
> +	/* Normal case */
> +	new = create_client(clname, dname);
> +	if (new == NULL) {
> +		status = nfserr_resource;
> +		goto out;
> +	}
> +
> +	copy_verf(new, &verf);
> +	copy_cred(&new->cl_cred, &rqstp->rq_cred);
> +	new->cl_addr = ip_addr;
> +	gen_clid(new);
> +	gen_confirm(new);
> +	add_to_unconfirmed(new, strhashval);
> +out_copy:
> +	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
> +	exid->clientid.cl_id = new->cl_clientid.cl_id;
> +
> +	new->cl_seqid = exid->seqid = 1;
> +	nfsd4_set_ex_flags(new, exid);
> +
> +	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
> +		new->cl_seqid, new->cl_exchange_flags);
> +	status = nfs_ok;
> +
> +out:
> +	nfs4_unlock_state();
> +error:
> +	dprintk("nfsd4_exchange_id returns %d\n", ntohl(status));
> +	return status;
>  }
>  
>  __be32
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index b082d07..840cf6a 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -45,6 +45,7 @@
>  #include <linux/fs.h>
>  #include <linux/namei.h>
>  #include <linux/vfs.h>
> +#include <linux/utsname.h>
>  #include <linux/sunrpc/xdr.h>
>  #include <linux/sunrpc/svc.h>
>  #include <linux/sunrpc/clnt.h>
> @@ -999,9 +1000,100 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
>  #if defined(CONFIG_NFSD_V4_1)
>  static __be32
>  nfsd4_decode_exchange_id(struct nfsd4_compoundargs *argp,
> -			 struct nfsd4_exchange_id *clid)
> +			 struct nfsd4_exchange_id *exid)
>  {
> -	return nfserr_opnotsupp;	/* stub */
> +	int dummy;
> +	DECODE_HEAD;
> +
> +	READ_BUF(NFS4_VERIFIER_SIZE);
> +	COPYMEM(exid->verifier.data, NFS4_VERIFIER_SIZE);
> +
> +	READ_BUF(4);
> +	READ32(exid->id_len);
> +
> +	READ_BUF(exid->id_len);
> +	SAVEMEM(exid->id, exid->id_len);
> +
> +	READ_BUF(4);
> +	READ32(exid->flags);
> +
> +	/* Ignore state_protect4_a */
> +	READ_BUF(4);
> +	READ32(exid->spa_how);
> +	switch (exid->spa_how) {
> +	case SP4_NONE:
> +		break;
> +	case SP4_MACH_CRED:
> +		/* spo_must_enforce */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +
> +		/* spo_must_allow */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +		break;
> +	case SP4_SSV:
> +		/* ssp_ops */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy * 4);
> +		p += dummy;
> +
> +		/* ssp_hash_algs<> */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* ssp_encr_algs<> */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* ssp_window and ssp_num_gss_handles */
> +		READ_BUF(8);
> +		READ32(dummy);
> +		READ32(dummy);
> +		break;
> +	default:
> +		goto xdr_error;
> +	}
> +
> +	/* Ignore Implementation ID */
> +	READ_BUF(4);    /* nfs_impl_id4 array length */
> +	READ32(dummy);
> +
> +	if (dummy > 1)
> +		goto xdr_error;
> +
> +	if (dummy == 1) {
> +		/* nii_domain */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* nii_name */
> +		READ_BUF(4);
> +		READ32(dummy);
> +		READ_BUF(dummy);
> +		p += XDR_QUADLEN(dummy);
> +
> +		/* nii_date */
> +		READ_BUF(12);
> +		p += 3;
> +	}
> +	DECODE_TAIL;
>  }
>  
>  static __be32
> @@ -2672,7 +2764,55 @@ static __be32
>  nfsd4_encode_exchange_id(struct nfsd4_compoundres *resp, int nfserr,
>  			 struct nfsd4_exchange_id *exid)
>  {
> -	/* stub */
> +	ENCODE_HEAD;
> +	char *major_id;
> +	char *server_scope;
> +	int major_id_sz;
> +	int server_scope_sz;
> +	uint64_t minor_id = 0;
> +
> +	if (nfserr)
> +		goto out;
> +
> +	major_id = utsname()->nodename;
> +	major_id_sz = strlen(major_id);
> +	server_scope = utsname()->nodename;
> +	server_scope_sz = strlen(server_scope);
> +
> +	RESERVE_SPACE(
> +		8 /* eir_clientid */ +
> +		4 /* eir_sequenceid */ +
> +		4 /* eir_flags */ +
> +		4 /* spr_how (SP4_NONE) */ +
> +		8 /* so_minor_id */ +
> +		4 /* so_major_id.len */ +
> +		(XDR_QUADLEN(major_id_sz) * 4) +
> +		4 /* eir_server_scope.len */ +
> +		(XDR_QUADLEN(server_scope_sz) * 4) +
> +		4 /* eir_server_impl_id.count (0) */);
> +
> +	WRITEMEM(&exid->clientid, 8);
> +	WRITE32(exid->seqid);
> +	WRITE32(exid->flags);
> +
> +	/* state_protect4_r. Currently only support SP4_NONE */
> +	BUG_ON(exid->spa_how != SP4_NONE);
> +	WRITE32(exid->spa_how);
> +
> +	/* The server_owner struct */
> +	WRITE64(minor_id);      /* Minor id */
> +	/* major id */
> +	WRITE32(major_id_sz);
> +	WRITEMEM(major_id, major_id_sz);
> +
> +	/* Server scope */
> +	WRITE32(server_scope_sz);
> +	WRITEMEM(server_scope, server_scope_sz);
> +
> +	/* Implementation id */
> +	WRITE32(0);	/* zero length nfs_impl_id4 array */
> +	ADJUST_ARGS();
> +out:
>  	return nfserr;
>  }
>  
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 7592d7b..5de36a7 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -173,6 +173,8 @@ struct nfs4_client {
>  	u32			cl_firststate;	/* recovery dir creation */
>  #ifdef CONFIG_NFSD_V4_1
>  	struct list_head	cl_sessions;
> +	u32			cl_seqid;       /* seqid for create_session */
> +	u32			cl_exchange_flags;
>  #endif /* CONFIG_NFSD_V4_1 */
>  };
>  
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 0148d54..ea5a427 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -348,7 +348,13 @@ struct nfsd4_write {
>  
>  #if defined(CONFIG_NFSD_V4_1)
>  struct nfsd4_exchange_id {
> -	int	foo;	/* stub */
> +	nfs4_verifier	verifier;
> +	u32		id_len;
> +	char		*id;
> +	u32		flags;
> +	clientid_t	clientid;
> +	u32		seqid;
> +	int		spa_how;
>  };
>  
>  struct nfsd4_create_session {
> -- 
> 1.6.2.1
> 

Re: [pnfs] [PATCH v2 15/47] nfsd41: exchange_id operation

[J. Bruce Fields]

> > +		if (ip_addr != conf->cl_addr &&
> > +		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A))
> {
> > +			/* Client collision. 18.35.4 case 3 */
> > +			status = nfserr_clid_inuse;
> > +			goto out;
> > +		}
> > +		/*
> > +		 * Set bit when the owner id and verifier map to an
> already
> > +		 * confirmed client id (18.35.3).
> > +		 */
> > +		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
> > +
> > +		/*
> > +		 * Falling into 18.35.4 case 2, possible router replay.
> 

I wrote:
> Checking the spec: case 2 says: "If the server has the following
> confirmed record, and the request does not have
> EXCHGID4_FLAG_UPD_CONFIRMED_REC_A set,..."
> 
> But that flag *is* set when we get to this code.
> 

On Tue, Mar 31, 2009 at 04:59:38PM -0400, Sager, Mike wrote:
> Isn't this case 6?
> 
> I don't think EXCHGID4_FLAG_UPD_CONFIRMED_REC_A is necessarily set when
> we get here.  If the verifier, credentials, and ip addresses match,  we
> can get here without the flag being set.  The comment should be updated,
> but I think it actually covers both cases 2 and 6 here with the code
> snippet under out_copy.  It doesn't seem to make sense to essentially
> ignore the value of the flag, but I think the end result is the
> same...at least based on my interpretation of the spec :)

OK, maybe--though in that case the ip_addr/exid->flags conditional
should probably just be removed entirely?

In any case someone needs to take a careful look at this function.

--b.

RE: [pnfs] [PATCH v2 15/47] nfsd41: exchange_id operation

[Sager, Mike]

Yes, I guess it should be removed.  It's in there in an attempt to
detect a client collision (case 3: same client owner, different ip, flag
not set), but as you and Benny have pointed out, basing it on an ip
check is flawed anyways...

Mike 

-----Original Message-----
From: J. Bruce Fields [mailto:bfields@fieldses.org] 
Sent: Wednesday, April 01, 2009 11:02 AM
To: Sager, Mike
Cc: Benny Halevy; linux-nfs@vger.kernel.org; pnfs@linux-nfs.org
Subject: Re: [pnfs] [PATCH v2 15/47] nfsd41: exchange_id operation


> > +		if (ip_addr != conf->cl_addr &&
> > +		    !(exid->flags & EXCHGID4_FLAG_UPD_CONFIRMED_REC_A))
> {
> > +			/* Client collision. 18.35.4 case 3 */
> > +			status = nfserr_clid_inuse;
> > +			goto out;
> > +		}
> > +		/*
> > +		 * Set bit when the owner id and verifier map to an
> already
> > +		 * confirmed client id (18.35.3).
> > +		 */
> > +		exid->flags |= EXCHGID4_FLAG_CONFIRMED_R;
> > +
> > +		/*
> > +		 * Falling into 18.35.4 case 2, possible router replay.
> 

I wrote:
> Checking the spec: case 2 says: "If the server has the following 
> confirmed record, and the request does not have 
> EXCHGID4_FLAG_UPD_CONFIRMED_REC_A set,..."
> 
> But that flag *is* set when we get to this code.
> 

On Tue, Mar 31, 2009 at 04:59:38PM -0400, Sager, Mike wrote:
> Isn't this case 6?
> 
> I don't think EXCHGID4_FLAG_UPD_CONFIRMED_REC_A is necessarily set 
> when we get here.  If the verifier, credentials, and ip addresses 
> match,  we can get here without the flag being set.  The comment 
> should be updated, but I think it actually covers both cases 2 and 6 
> here with the code snippet under out_copy.  It doesn't seem to make 
> sense to essentially ignore the value of the flag, but I think the end

> result is the same...at least based on my interpretation of the spec 
> :)

OK, maybe--though in that case the ip_addr/exid->flags conditional
should probably just be removed entirely?

In any case someone needs to take a careful look at this function.

--b.

[PATCH v2 16/47] nfsd41: match clientid establishment method

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

We need to distinguish between client names provided by NFSv4.0 clients
SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
up the clientid by string.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
[nfsd41: use boolean values for use_exchange_id argument]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4recover.c      |    3 ++-
 fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
 include/linux/nfsd/state.h |    2 +-
 3 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index b11cf8d..3444c00 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
 {
 	int status;
 
-	if (nfs4_has_reclaimed_state(child->d_name.name))
+	/* note: we currently use this path only for minorversion 0 */
+	if (nfs4_has_reclaimed_state(child->d_name.name, false))
 		return 0;
 
 	status = nfsd4_clear_clid_dir(parent, child);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 09c63ff..0c39376 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
 	return NULL;
 }
 
+/*
+ * Return 1 iff clp's clientid establishment method matches the use_exchange_id
+ * parameter. Matching is based on the fact the at least one of the
+ * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
+ */
+static inline int
+match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
+{
+#if defined(CONFIG_NFSD_V4_1)
+	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
+#else  /* CONFIG_NFSD_V4_1 */
+	return 1;
+#endif /* CONFIG_NFSD_V4_1 */
+}
+
 static struct nfs4_client *
-find_confirmed_client_by_str(const char *dname, unsigned int hashval)
+find_confirmed_client_by_str(const char *dname, unsigned int hashval,
+			     bool use_exchange_id)
 {
 	struct nfs4_client *clp;
 
 	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
-		if (same_name(clp->cl_recdir, dname))
+		if (same_name(clp->cl_recdir, dname) &&
+		    match_clientid_establishment(clp, use_exchange_id))
 			return clp;
 	}
 	return NULL;
 }
 
 static struct nfs4_client *
-find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
+find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
+			       bool use_exchange_id)
 {
 	struct nfs4_client *clp;
 
 	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
-		if (same_name(clp->cl_recdir, dname))
+		if (same_name(clp->cl_recdir, dname) &&
+		    match_clientid_establishment(clp, use_exchange_id))
 			return clp;
 	}
 	return NULL;
@@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
 	nfs4_lock_state();
 	status = nfs_ok;
 
-	conf = find_confirmed_client_by_str(dname, strhashval);
+	conf = find_confirmed_client_by_str(dname, strhashval, true);
 	if (conf) {
 		if (!same_verf(&verf, &conf->cl_verifier)) {
 			/* 18.35.4 case 8 */
@@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
 		}
 	}
 
-	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
+	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
 	if (unconf) {
 		/*
 		 * Possible retry or client restart.  Per 18.35.4 case 4,
@@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	strhashval = clientstr_hashval(dname);
 
 	nfs4_lock_state();
-	conf = find_confirmed_client_by_str(dname, strhashval);
+	conf = find_confirmed_client_by_str(dname, strhashval, false);
 	if (conf) {
 		/* RFC 3530 14.2.33 CASE 0: */
 		status = nfserr_clid_inuse;
@@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	 * has a description of SETCLIENTID request processing consisting
 	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
 	 */
-	unconf = find_unconfirmed_client_by_str(dname, strhashval);
+	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
 	status = nfserr_resource;
 	if (!conf) {
 		/*
@@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 			unsigned int hash =
 				clientstr_hashval(unconf->cl_recdir);
 			conf = find_confirmed_client_by_str(unconf->cl_recdir,
-									hash);
+							    hash, false);
 			if (conf) {
 				nfsd4_remove_clid_dir(conf);
 				expire_client(conf);
@@ -3332,12 +3351,12 @@ alloc_reclaim(void)
 }
 
 int
-nfs4_has_reclaimed_state(const char *name)
+nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
 {
 	unsigned int strhashval = clientstr_hashval(name);
 	struct nfs4_client *clp;
 
-	clp = find_confirmed_client_by_str(name, strhashval);
+	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
 	return clp ? 1 : 0;
 }
 
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 5de36a7..feab6ec 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
 extern int nfsd4_recdir_load(void);
 extern void nfsd4_shutdown_recdir(void);
 extern int nfs4_client_to_reclaim(const char *name);
-extern int nfs4_has_reclaimed_state(const char *name);
+extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
 extern void nfsd4_recdir_purge_old(void);
 extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
 extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
-- 
1.6.2.1

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> We need to distinguish between client names provided by NFSv4.0 clients
> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
> up the clientid by string.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> Signed-off-by: Andy Adamson <andros@netapp.com>
> [nfsd41: use boolean values for use_exchange_id argument]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4recover.c      |    3 ++-
>  fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
>  include/linux/nfsd/state.h |    2 +-
>  3 files changed, 33 insertions(+), 13 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
> index b11cf8d..3444c00 100644
> --- a/fs/nfsd/nfs4recover.c
> +++ b/fs/nfsd/nfs4recover.c
> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
>  {
>  	int status;
>  
> -	if (nfs4_has_reclaimed_state(child->d_name.name))
> +	/* note: we currently use this path only for minorversion 0 */
> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
>  		return 0;
>  
>  	status = nfsd4_clear_clid_dir(parent, child);
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 09c63ff..0c39376 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
>  	return NULL;
>  }
>  
> +/*
> + * Return 1 iff clp's clientid establishment method matches the use_exchange_id
> + * parameter. Matching is based on the fact the at least one of the
> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
> + */
> +static inline int
> +match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
> +{
> +#if defined(CONFIG_NFSD_V4_1)
> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);

This seems a bit baroque; the "!= false" is a no-op, for one thing,
isn't it?

--b.

> +#else  /* CONFIG_NFSD_V4_1 */
> +	return 1;
> +#endif /* CONFIG_NFSD_V4_1 */
> +}
> +
>  static struct nfs4_client *
> -find_confirmed_client_by_str(const char *dname, unsigned int hashval)
> +find_confirmed_client_by_str(const char *dname, unsigned int hashval,
> +			     bool use_exchange_id)
>  {
>  	struct nfs4_client *clp;
>  
>  	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
> -		if (same_name(clp->cl_recdir, dname))
> +		if (same_name(clp->cl_recdir, dname) &&
> +		    match_clientid_establishment(clp, use_exchange_id))
>  			return clp;
>  	}
>  	return NULL;
>  }
>  
>  static struct nfs4_client *
> -find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
> +find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
> +			       bool use_exchange_id)
>  {
>  	struct nfs4_client *clp;
>  
>  	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
> -		if (same_name(clp->cl_recdir, dname))
> +		if (same_name(clp->cl_recdir, dname) &&
> +		    match_clientid_establishment(clp, use_exchange_id))
>  			return clp;
>  	}
>  	return NULL;
> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>  	nfs4_lock_state();
>  	status = nfs_ok;
>  
> -	conf = find_confirmed_client_by_str(dname, strhashval);
> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
>  	if (conf) {
>  		if (!same_verf(&verf, &conf->cl_verifier)) {
>  			/* 18.35.4 case 8 */
> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>  		}
>  	}
>  
> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
>  	if (unconf) {
>  		/*
>  		 * Possible retry or client restart.  Per 18.35.4 case 4,
> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	strhashval = clientstr_hashval(dname);
>  
>  	nfs4_lock_state();
> -	conf = find_confirmed_client_by_str(dname, strhashval);
> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
>  	if (conf) {
>  		/* RFC 3530 14.2.33 CASE 0: */
>  		status = nfserr_clid_inuse;
> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	 * has a description of SETCLIENTID request processing consisting
>  	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
>  	 */
> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
>  	status = nfserr_resource;
>  	if (!conf) {
>  		/*
> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>  			unsigned int hash =
>  				clientstr_hashval(unconf->cl_recdir);
>  			conf = find_confirmed_client_by_str(unconf->cl_recdir,
> -									hash);
> +							    hash, false);
>  			if (conf) {
>  				nfsd4_remove_clid_dir(conf);
>  				expire_client(conf);
> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
>  }
>  
>  int
> -nfs4_has_reclaimed_state(const char *name)
> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
>  {
>  	unsigned int strhashval = clientstr_hashval(name);
>  	struct nfs4_client *clp;
>  
> -	clp = find_confirmed_client_by_str(name, strhashval);
> +	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
>  	return clp ? 1 : 0;
>  }
>  
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 5de36a7..feab6ec 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
>  extern int nfsd4_recdir_load(void);
>  extern void nfsd4_shutdown_recdir(void);
>  extern int nfs4_client_to_reclaim(const char *name);
> -extern int nfs4_has_reclaimed_state(const char *name);
> +extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
>  extern void nfsd4_recdir_purge_old(void);
>  extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
>  extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[Benny Halevy]

On Mar. 31, 2009, 1:07 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> We need to distinguish between client names provided by NFSv4.0 clients
>> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
>> up the clientid by string.
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> [nfsd41: use boolean values for use_exchange_id argument]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4recover.c      |    3 ++-
>>  fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
>>  include/linux/nfsd/state.h |    2 +-
>>  3 files changed, 33 insertions(+), 13 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
>> index b11cf8d..3444c00 100644
>> --- a/fs/nfsd/nfs4recover.c
>> +++ b/fs/nfsd/nfs4recover.c
>> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
>>  {
>>  	int status;
>>  
>> -	if (nfs4_has_reclaimed_state(child->d_name.name))
>> +	/* note: we currently use this path only for minorversion 0 */
>> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
>>  		return 0;
>>  
>>  	status = nfsd4_clear_clid_dir(parent, child);
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index 09c63ff..0c39376 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
>>  	return NULL;
>>  }
>>  
>> +/*
>> + * Return 1 iff clp's clientid establishment method matches the use_exchange_id
>> + * parameter. Matching is based on the fact the at least one of the
>> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
>> + */
>> +static inline int
>> +match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
>> +{
>> +#if defined(CONFIG_NFSD_V4_1)
>> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
> 
> This seems a bit baroque; the "!= false" is a no-op, for one thing,
> isn't it?

The idea is to efficiently do explicit boolean comparison.
Without the "!= false" the comparison will fail for
1 != use_exchange_id != 0.

Does this make it any clearer:

	if (use_exchange_flags)
		return clp->cl_exchange_flags != 0;
	return clp->cl_exchange_flags == 0;

or maybe

	bool has_exchange_flags = (clp->cl_exchange_flags != 0);
	if (use_exchange_flags)
		return has_exchange_flags;
	return !has_exchange_flags;

or a long boolean expression

	return ((clp->cl_exchange_flags && use_exchange_id) ||
		(!clp->cl_exchange_flags && !use_exchange_id))

Benny

> 
> --b.
> 
>> +#else  /* CONFIG_NFSD_V4_1 */
>> +	return 1;
>> +#endif /* CONFIG_NFSD_V4_1 */
>> +}
>> +
>>  static struct nfs4_client *
>> -find_confirmed_client_by_str(const char *dname, unsigned int hashval)
>> +find_confirmed_client_by_str(const char *dname, unsigned int hashval,
>> +			     bool use_exchange_id)
>>  {
>>  	struct nfs4_client *clp;
>>  
>>  	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
>> -		if (same_name(clp->cl_recdir, dname))
>> +		if (same_name(clp->cl_recdir, dname) &&
>> +		    match_clientid_establishment(clp, use_exchange_id))
>>  			return clp;
>>  	}
>>  	return NULL;
>>  }
>>  
>>  static struct nfs4_client *
>> -find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
>> +find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
>> +			       bool use_exchange_id)
>>  {
>>  	struct nfs4_client *clp;
>>  
>>  	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
>> -		if (same_name(clp->cl_recdir, dname))
>> +		if (same_name(clp->cl_recdir, dname) &&
>> +		    match_clientid_establishment(clp, use_exchange_id))
>>  			return clp;
>>  	}
>>  	return NULL;
>> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>  	nfs4_lock_state();
>>  	status = nfs_ok;
>>  
>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
>>  	if (conf) {
>>  		if (!same_verf(&verf, &conf->cl_verifier)) {
>>  			/* 18.35.4 case 8 */
>> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>  		}
>>  	}
>>  
>> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
>> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
>>  	if (unconf) {
>>  		/*
>>  		 * Possible retry or client restart.  Per 18.35.4 case 4,
>> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>  	strhashval = clientstr_hashval(dname);
>>  
>>  	nfs4_lock_state();
>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
>>  	if (conf) {
>>  		/* RFC 3530 14.2.33 CASE 0: */
>>  		status = nfserr_clid_inuse;
>> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>  	 * has a description of SETCLIENTID request processing consisting
>>  	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
>>  	 */
>> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
>> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
>>  	status = nfserr_resource;
>>  	if (!conf) {
>>  		/*
>> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>>  			unsigned int hash =
>>  				clientstr_hashval(unconf->cl_recdir);
>>  			conf = find_confirmed_client_by_str(unconf->cl_recdir,
>> -									hash);
>> +							    hash, false);
>>  			if (conf) {
>>  				nfsd4_remove_clid_dir(conf);
>>  				expire_client(conf);
>> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
>>  }
>>  
>>  int
>> -nfs4_has_reclaimed_state(const char *name)
>> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
>>  {
>>  	unsigned int strhashval = clientstr_hashval(name);
>>  	struct nfs4_client *clp;
>>  
>> -	clp = find_confirmed_client_by_str(name, strhashval);
>> +	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
>>  	return clp ? 1 : 0;
>>  }
>>  
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index 5de36a7..feab6ec 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
>>  extern int nfsd4_recdir_load(void);
>>  extern void nfsd4_shutdown_recdir(void);
>>  extern int nfs4_client_to_reclaim(const char *name);
>> -extern int nfs4_has_reclaimed_state(const char *name);
>> +extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
>>  extern void nfsd4_recdir_purge_old(void);
>>  extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
>>  extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
>> -- 
>> 1.6.2.1
>>

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[J. Bruce Fields]

On Tue, Mar 31, 2009 at 09:59:17AM +0300, Benny Halevy wrote:
> On Mar. 31, 2009, 1:07 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
> >> From: Andy Adamson <andros@netapp.com>
> >>
> >> We need to distinguish between client names provided by NFSv4.0 clients
> >> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
> >> up the clientid by string.
> >>
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> Signed-off-by: Andy Adamson <andros@netapp.com>
> >> [nfsd41: use boolean values for use_exchange_id argument]
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> ---
> >>  fs/nfsd/nfs4recover.c      |    3 ++-
> >>  fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
> >>  include/linux/nfsd/state.h |    2 +-
> >>  3 files changed, 33 insertions(+), 13 deletions(-)
> >>
> >> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
> >> index b11cf8d..3444c00 100644
> >> --- a/fs/nfsd/nfs4recover.c
> >> +++ b/fs/nfsd/nfs4recover.c
> >> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
> >>  {
> >>  	int status;
> >>  
> >> -	if (nfs4_has_reclaimed_state(child->d_name.name))
> >> +	/* note: we currently use this path only for minorversion 0 */
> >> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
> >>  		return 0;
> >>  
> >>  	status = nfsd4_clear_clid_dir(parent, child);
> >> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> >> index 09c63ff..0c39376 100644
> >> --- a/fs/nfsd/nfs4state.c
> >> +++ b/fs/nfsd/nfs4state.c
> >> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
> >>  	return NULL;
> >>  }
> >>  
> >> +/*
> >> + * Return 1 iff clp's clientid establishment method matches the use_exchange_id
> >> + * parameter. Matching is based on the fact the at least one of the
> >> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
> >> + */
> >> +static inline int
> >> +match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
> >> +{
> >> +#if defined(CONFIG_NFSD_V4_1)
> >> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
> > 
> > This seems a bit baroque; the "!= false" is a no-op, for one thing,
> > isn't it?
> 
> The idea is to efficiently do explicit boolean comparison.
> Without the "!= false" the comparison will fail for
> 1 != use_exchange_id != 0.

use_exchange_id is defined to be boolean.

--b.

> 
> Does this make it any clearer:
> 
> 	if (use_exchange_flags)
> 		return clp->cl_exchange_flags != 0;
> 	return clp->cl_exchange_flags == 0;
> 
> or maybe
> 
> 	bool has_exchange_flags = (clp->cl_exchange_flags != 0);
> 	if (use_exchange_flags)
> 		return has_exchange_flags;
> 	return !has_exchange_flags;
> 
> or a long boolean expression
> 
> 	return ((clp->cl_exchange_flags && use_exchange_id) ||
> 		(!clp->cl_exchange_flags && !use_exchange_id))
> 
> Benny
> 
> > 
> > --b.
> > 
> >> +#else  /* CONFIG_NFSD_V4_1 */
> >> +	return 1;
> >> +#endif /* CONFIG_NFSD_V4_1 */
> >> +}
> >> +
> >>  static struct nfs4_client *
> >> -find_confirmed_client_by_str(const char *dname, unsigned int hashval)
> >> +find_confirmed_client_by_str(const char *dname, unsigned int hashval,
> >> +			     bool use_exchange_id)
> >>  {
> >>  	struct nfs4_client *clp;
> >>  
> >>  	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
> >> -		if (same_name(clp->cl_recdir, dname))
> >> +		if (same_name(clp->cl_recdir, dname) &&
> >> +		    match_clientid_establishment(clp, use_exchange_id))
> >>  			return clp;
> >>  	}
> >>  	return NULL;
> >>  }
> >>  
> >>  static struct nfs4_client *
> >> -find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
> >> +find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
> >> +			       bool use_exchange_id)
> >>  {
> >>  	struct nfs4_client *clp;
> >>  
> >>  	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
> >> -		if (same_name(clp->cl_recdir, dname))
> >> +		if (same_name(clp->cl_recdir, dname) &&
> >> +		    match_clientid_establishment(clp, use_exchange_id))
> >>  			return clp;
> >>  	}
> >>  	return NULL;
> >> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
> >>  	nfs4_lock_state();
> >>  	status = nfs_ok;
> >>  
> >> -	conf = find_confirmed_client_by_str(dname, strhashval);
> >> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
> >>  	if (conf) {
> >>  		if (!same_verf(&verf, &conf->cl_verifier)) {
> >>  			/* 18.35.4 case 8 */
> >> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
> >>  		}
> >>  	}
> >>  
> >> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
> >> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
> >>  	if (unconf) {
> >>  		/*
> >>  		 * Possible retry or client restart.  Per 18.35.4 case 4,
> >> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> >>  	strhashval = clientstr_hashval(dname);
> >>  
> >>  	nfs4_lock_state();
> >> -	conf = find_confirmed_client_by_str(dname, strhashval);
> >> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
> >>  	if (conf) {
> >>  		/* RFC 3530 14.2.33 CASE 0: */
> >>  		status = nfserr_clid_inuse;
> >> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> >>  	 * has a description of SETCLIENTID request processing consisting
> >>  	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
> >>  	 */
> >> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
> >> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
> >>  	status = nfserr_resource;
> >>  	if (!conf) {
> >>  		/*
> >> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
> >>  			unsigned int hash =
> >>  				clientstr_hashval(unconf->cl_recdir);
> >>  			conf = find_confirmed_client_by_str(unconf->cl_recdir,
> >> -									hash);
> >> +							    hash, false);
> >>  			if (conf) {
> >>  				nfsd4_remove_clid_dir(conf);
> >>  				expire_client(conf);
> >> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
> >>  }
> >>  
> >>  int
> >> -nfs4_has_reclaimed_state(const char *name)
> >> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
> >>  {
> >>  	unsigned int strhashval = clientstr_hashval(name);
> >>  	struct nfs4_client *clp;
> >>  
> >> -	clp = find_confirmed_client_by_str(name, strhashval);
> >> +	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
> >>  	return clp ? 1 : 0;
> >>  }
> >>  
> >> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> >> index 5de36a7..feab6ec 100644
> >> --- a/include/linux/nfsd/state.h
> >> +++ b/include/linux/nfsd/state.h
> >> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
> >>  extern int nfsd4_recdir_load(void);
> >>  extern void nfsd4_shutdown_recdir(void);
> >>  extern int nfs4_client_to_reclaim(const char *name);
> >> -extern int nfs4_has_reclaimed_state(const char *name);
> >> +extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
> >>  extern void nfsd4_recdir_purge_old(void);
> >>  extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
> >>  extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
> >> -- 
> >> 1.6.2.1
> >>

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[Benny Halevy]

On Mar. 31, 2009, 20:59 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Tue, Mar 31, 2009 at 09:59:17AM +0300, Benny Halevy wrote:
>> On Mar. 31, 2009, 1:07 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
>>>> From: Andy Adamson <andros@netapp.com>
>>>>
>>>> We need to distinguish between client names provided by NFSv4.0 clients
>>>> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
>>>> up the clientid by string.
>>>>
>>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>>> [nfsd41: use boolean values for use_exchange_id argument]
>>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>>> ---
>>>>  fs/nfsd/nfs4recover.c      |    3 ++-
>>>>  fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
>>>>  include/linux/nfsd/state.h |    2 +-
>>>>  3 files changed, 33 insertions(+), 13 deletions(-)
>>>>
>>>> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
>>>> index b11cf8d..3444c00 100644
>>>> --- a/fs/nfsd/nfs4recover.c
>>>> +++ b/fs/nfsd/nfs4recover.c
>>>> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
>>>>  {
>>>>  	int status;
>>>>  
>>>> -	if (nfs4_has_reclaimed_state(child->d_name.name))
>>>> +	/* note: we currently use this path only for minorversion 0 */
>>>> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
>>>>  		return 0;
>>>>  
>>>>  	status = nfsd4_clear_clid_dir(parent, child);
>>>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>>>> index 09c63ff..0c39376 100644
>>>> --- a/fs/nfsd/nfs4state.c
>>>> +++ b/fs/nfsd/nfs4state.c
>>>> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
>>>>  	return NULL;
>>>>  }
>>>>  
>>>> +/*
>>>> + * Return 1 iff clp's clientid establishment method matches the use_exchange_id
>>>> + * parameter. Matching is based on the fact the at least one of the
>>>> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
>>>> + */
>>>> +static inline int
>>>> +match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
>>>> +{
>>>> +#if defined(CONFIG_NFSD_V4_1)
>>>> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
>>> This seems a bit baroque; the "!= false" is a no-op, for one thing,
>>> isn't it?
>> The idea is to efficiently do explicit boolean comparison.
>> Without the "!= false" the comparison will fail for
>> 1 != use_exchange_id != 0.
> 
> use_exchange_id is defined to be boolean.

Whoops.  You're right.
For some reason I thought bool was just a typedef'ed int.
I see that gcc's _Bool is actually doing the right thing with it.

Benny

> 
> --b.
> 
>> Does this make it any clearer:
>>
>> 	if (use_exchange_flags)
>> 		return clp->cl_exchange_flags != 0;
>> 	return clp->cl_exchange_flags == 0;
>>
>> or maybe
>>
>> 	bool has_exchange_flags = (clp->cl_exchange_flags != 0);
>> 	if (use_exchange_flags)
>> 		return has_exchange_flags;
>> 	return !has_exchange_flags;
>>
>> or a long boolean expression
>>
>> 	return ((clp->cl_exchange_flags && use_exchange_id) ||
>> 		(!clp->cl_exchange_flags && !use_exchange_id))
>>
>> Benny
>>
>>> --b.
>>>
>>>> +#else  /* CONFIG_NFSD_V4_1 */
>>>> +	return 1;
>>>> +#endif /* CONFIG_NFSD_V4_1 */
>>>> +}
>>>> +
>>>>  static struct nfs4_client *
>>>> -find_confirmed_client_by_str(const char *dname, unsigned int hashval)
>>>> +find_confirmed_client_by_str(const char *dname, unsigned int hashval,
>>>> +			     bool use_exchange_id)
>>>>  {
>>>>  	struct nfs4_client *clp;
>>>>  
>>>>  	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
>>>> -		if (same_name(clp->cl_recdir, dname))
>>>> +		if (same_name(clp->cl_recdir, dname) &&
>>>> +		    match_clientid_establishment(clp, use_exchange_id))
>>>>  			return clp;
>>>>  	}
>>>>  	return NULL;
>>>>  }
>>>>  
>>>>  static struct nfs4_client *
>>>> -find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
>>>> +find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
>>>> +			       bool use_exchange_id)
>>>>  {
>>>>  	struct nfs4_client *clp;
>>>>  
>>>>  	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
>>>> -		if (same_name(clp->cl_recdir, dname))
>>>> +		if (same_name(clp->cl_recdir, dname) &&
>>>> +		    match_clientid_establishment(clp, use_exchange_id))
>>>>  			return clp;
>>>>  	}
>>>>  	return NULL;
>>>> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>>>  	nfs4_lock_state();
>>>>  	status = nfs_ok;
>>>>  
>>>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>>>> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
>>>>  	if (conf) {
>>>>  		if (!same_verf(&verf, &conf->cl_verifier)) {
>>>>  			/* 18.35.4 case 8 */
>>>> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>>>  		}
>>>>  	}
>>>>  
>>>> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
>>>> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
>>>>  	if (unconf) {
>>>>  		/*
>>>>  		 * Possible retry or client restart.  Per 18.35.4 case 4,
>>>> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>>>  	strhashval = clientstr_hashval(dname);
>>>>  
>>>>  	nfs4_lock_state();
>>>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>>>> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
>>>>  	if (conf) {
>>>>  		/* RFC 3530 14.2.33 CASE 0: */
>>>>  		status = nfserr_clid_inuse;
>>>> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>>>  	 * has a description of SETCLIENTID request processing consisting
>>>>  	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
>>>>  	 */
>>>> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
>>>> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
>>>>  	status = nfserr_resource;
>>>>  	if (!conf) {
>>>>  		/*
>>>> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>>>>  			unsigned int hash =
>>>>  				clientstr_hashval(unconf->cl_recdir);
>>>>  			conf = find_confirmed_client_by_str(unconf->cl_recdir,
>>>> -									hash);
>>>> +							    hash, false);
>>>>  			if (conf) {
>>>>  				nfsd4_remove_clid_dir(conf);
>>>>  				expire_client(conf);
>>>> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
>>>>  }
>>>>  
>>>>  int
>>>> -nfs4_has_reclaimed_state(const char *name)
>>>> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
>>>>  {
>>>>  	unsigned int strhashval = clientstr_hashval(name);
>>>>  	struct nfs4_client *clp;
>>>>  
>>>> -	clp = find_confirmed_client_by_str(name, strhashval);
>>>> +	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
>>>>  	return clp ? 1 : 0;
>>>>  }
>>>>  
>>>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>>>> index 5de36a7..feab6ec 100644
>>>> --- a/include/linux/nfsd/state.h
>>>> +++ b/include/linux/nfsd/state.h
>>>> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
>>>>  extern int nfsd4_recdir_load(void);
>>>>  extern void nfsd4_shutdown_recdir(void);
>>>>  extern int nfs4_client_to_reclaim(const char *name);
>>>> -extern int nfs4_has_reclaimed_state(const char *name);
>>>> +extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
>>>>  extern void nfsd4_recdir_purge_old(void);
>>>>  extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
>>>>  extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
>>>> -- 
>>>> 1.6.2.1
>>>>

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> We need to distinguish between client names provided by NFSv4.0 clients
> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
> up the clientid by string.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> Signed-off-by: Andy Adamson <andros@netapp.com>
> [nfsd41: use boolean values for use_exchange_id argument]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4recover.c      |    3 ++-
>  fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
>  include/linux/nfsd/state.h |    2 +-
>  3 files changed, 33 insertions(+), 13 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
> index b11cf8d..3444c00 100644
> --- a/fs/nfsd/nfs4recover.c
> +++ b/fs/nfsd/nfs4recover.c
> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
>  {
>  	int status;
>  
> -	if (nfs4_has_reclaimed_state(child->d_name.name))
> +	/* note: we currently use this path only for minorversion 0 */

Why is that?

> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
>  		return 0;
>  
>  	status = nfsd4_clear_clid_dir(parent, child);
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 09c63ff..0c39376 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
>  	return NULL;
>  }
>  
> +/*
> + * Return 1 iff clp's clientid establishment method matches the use_exchange_id
> + * parameter. Matching is based on the fact the at least one of the
> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
> + */
> +static inline int
> +match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
> +{
> +#if defined(CONFIG_NFSD_V4_1)
> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
> +#else  /* CONFIG_NFSD_V4_1 */
> +	return 1;
> +#endif /* CONFIG_NFSD_V4_1 */
> +}

If the point is just to ensure that clients only match clients of the
same minorversion, why not just call this match_client_minorversion()?
You could still use cl_exchange_flags as the way to distinguish 4.0 from
4.1, but hide that detail away here.  In which case clearer might be:

	static inline u32 client_minorversion(struct nfs4_client *clp)
	{
		/*
		 * Note 4.1 clients always have one of
		 * EXCHGID4_FLAG_USE{NON_PNFS,PNFS_MDS,PNFS_DS} set.
		 */
		return clp->cl_exchange_flags != 0;
	}

	static inline int client_same_minorversion(nfs4_client *clp, u32 minorversion)
	{
		return client_minorversion(clp) == minorversion;
	}

or even just open-code the latter.

But: are the 4.0 and 4.1 client owner-name namespaces actually meant to
be distinct?  2.4.1 has me a bit confused here.

--b.

> +
>  static struct nfs4_client *
> -find_confirmed_client_by_str(const char *dname, unsigned int hashval)
> +find_confirmed_client_by_str(const char *dname, unsigned int hashval,
> +			     bool use_exchange_id)
>  {
>  	struct nfs4_client *clp;
>  
>  	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
> -		if (same_name(clp->cl_recdir, dname))
> +		if (same_name(clp->cl_recdir, dname) &&
> +		    match_clientid_establishment(clp, use_exchange_id))
>  			return clp;
>  	}
>  	return NULL;
>  }
>  
>  static struct nfs4_client *
> -find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
> +find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
> +			       bool use_exchange_id)
>  {
>  	struct nfs4_client *clp;
>  
>  	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
> -		if (same_name(clp->cl_recdir, dname))
> +		if (same_name(clp->cl_recdir, dname) &&
> +		    match_clientid_establishment(clp, use_exchange_id))
>  			return clp;
>  	}
>  	return NULL;
> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>  	nfs4_lock_state();
>  	status = nfs_ok;
>  
> -	conf = find_confirmed_client_by_str(dname, strhashval);
> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
>  	if (conf) {
>  		if (!same_verf(&verf, &conf->cl_verifier)) {
>  			/* 18.35.4 case 8 */
> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>  		}
>  	}
>  
> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
>  	if (unconf) {
>  		/*
>  		 * Possible retry or client restart.  Per 18.35.4 case 4,
> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	strhashval = clientstr_hashval(dname);
>  
>  	nfs4_lock_state();
> -	conf = find_confirmed_client_by_str(dname, strhashval);
> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
>  	if (conf) {
>  		/* RFC 3530 14.2.33 CASE 0: */
>  		status = nfserr_clid_inuse;
> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	 * has a description of SETCLIENTID request processing consisting
>  	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
>  	 */
> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
>  	status = nfserr_resource;
>  	if (!conf) {
>  		/*
> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>  			unsigned int hash =
>  				clientstr_hashval(unconf->cl_recdir);
>  			conf = find_confirmed_client_by_str(unconf->cl_recdir,
> -									hash);
> +							    hash, false);
>  			if (conf) {
>  				nfsd4_remove_clid_dir(conf);
>  				expire_client(conf);
> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
>  }
>  
>  int
> -nfs4_has_reclaimed_state(const char *name)
> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
>  {
>  	unsigned int strhashval = clientstr_hashval(name);
>  	struct nfs4_client *clp;
>  
> -	clp = find_confirmed_client_by_str(name, strhashval);
> +	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
>  	return clp ? 1 : 0;
>  }
>  
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 5de36a7..feab6ec 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
>  extern int nfsd4_recdir_load(void);
>  extern void nfsd4_shutdown_recdir(void);
>  extern int nfs4_client_to_reclaim(const char *name);
> -extern int nfs4_has_reclaimed_state(const char *name);
> +extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
>  extern void nfsd4_recdir_purge_old(void);
>  extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
>  extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[Benny Halevy]

On Mar. 31, 2009, 6:04 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> We need to distinguish between client names provided by NFSv4.0 clients
>> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when looking
>> up the clientid by string.
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> [nfsd41: use boolean values for use_exchange_id argument]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4recover.c      |    3 ++-
>>  fs/nfsd/nfs4state.c        |   41 ++++++++++++++++++++++++++++++-----------
>>  include/linux/nfsd/state.h |    2 +-
>>  3 files changed, 33 insertions(+), 13 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
>> index b11cf8d..3444c00 100644
>> --- a/fs/nfsd/nfs4recover.c
>> +++ b/fs/nfsd/nfs4recover.c
>> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry *child)
>>  {
>>  	int status;
>>  
>> -	if (nfs4_has_reclaimed_state(child->d_name.name))
>> +	/* note: we currently use this path only for minorversion 0 */
> 
> Why is that?

Hmm, I'm not sure this is true anymore.
Andy, do you recall?
One thing for sure, we currently implemented nothing to
propagate the "use_exchange_id" state onto the state
recovery mechanisms, so this comment merely reflects
that, though it isn't clear what "this path" means
in this context, i.e. is this the path we were called
in, or the path we're calling.

At any rate, if this is something we need to fix for 4.1
and it does not introduce any regression to 4.0, and if
the fix isn't trivial/simple, I suggest we add a FIXME comment,
and add it to our todo list to defer the solution post
this push effort.

> 
>> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
>>  		return 0;
>>  
>>  	status = nfsd4_clear_clid_dir(parent, child);
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index 09c63ff..0c39376 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
>>  	return NULL;
>>  }
>>  
>> +/*
>> + * Return 1 iff clp's clientid establishment method matches the use_exchange_id
>> + * parameter. Matching is based on the fact the at least one of the
>> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be set for v4.1
>> + */
>> +static inline int
>> +match_clientid_establishment(struct nfs4_client *clp, bool use_exchange_id)
>> +{
>> +#if defined(CONFIG_NFSD_V4_1)
>> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id != false);
>> +#else  /* CONFIG_NFSD_V4_1 */
>> +	return 1;
>> +#endif /* CONFIG_NFSD_V4_1 */
>> +}
> 
> If the point is just to ensure that clients only match clients of the
> same minorversion, why not just call this match_client_minorversion()?
> You could still use cl_exchange_flags as the way to distinguish 4.0 from
> 4.1, but hide that detail away here.  In which case clearer might be:
> 
> 	static inline u32 client_minorversion(struct nfs4_client *clp)
> 	{
> 		/*
> 		 * Note 4.1 clients always have one of
> 		 * EXCHGID4_FLAG_USE{NON_PNFS,PNFS_MDS,PNFS_DS} set.
> 		 */
> 		return clp->cl_exchange_flags != 0;
> 	}
> 
> 	static inline int client_same_minorversion(nfs4_client *clp, u32 minorversion)
> 	{
> 		return client_minorversion(clp) == minorversion;
> 	}
> 
> or even just open-code the latter.

I don't like using "minorversion" here since it is a numeric attribute
and may be larger than 1 in the future.  What we care about here is
whether the clientid was established via EXCHANGE_ID or via nfsv4.0
SET_CLIENTID et al, therefore we used cl_exchange_flags as an indication.

> 
> But: are the 4.0 and 4.1 client owner-name namespaces actually meant to
> be distinct?  2.4.1 has me a bit confused here.

This case is not implemented yet (note to self - update Doc)
If we implement it, and deal with v4.1 -> v4.1 downgrade (or prevention
thereof), then I think we can indeed unify the clientid spaces.
However, 2.4.1 is optional and we don't have to implement it right now.

Benny

> 
> --b.
> 
>> +
>>  static struct nfs4_client *
>> -find_confirmed_client_by_str(const char *dname, unsigned int hashval)
>> +find_confirmed_client_by_str(const char *dname, unsigned int hashval,
>> +			     bool use_exchange_id)
>>  {
>>  	struct nfs4_client *clp;
>>  
>>  	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
>> -		if (same_name(clp->cl_recdir, dname))
>> +		if (same_name(clp->cl_recdir, dname) &&
>> +		    match_clientid_establishment(clp, use_exchange_id))
>>  			return clp;
>>  	}
>>  	return NULL;
>>  }
>>  
>>  static struct nfs4_client *
>> -find_unconfirmed_client_by_str(const char *dname, unsigned int hashval)
>> +find_unconfirmed_client_by_str(const char *dname, unsigned int hashval,
>> +			       bool use_exchange_id)
>>  {
>>  	struct nfs4_client *clp;
>>  
>>  	list_for_each_entry(clp, &unconf_str_hashtbl[hashval], cl_strhash) {
>> -		if (same_name(clp->cl_recdir, dname))
>> +		if (same_name(clp->cl_recdir, dname) &&
>> +		    match_clientid_establishment(clp, use_exchange_id))
>>  			return clp;
>>  	}
>>  	return NULL;
>> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>  	nfs4_lock_state();
>>  	status = nfs_ok;
>>  
>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
>>  	if (conf) {
>>  		if (!same_verf(&verf, &conf->cl_verifier)) {
>>  			/* 18.35.4 case 8 */
>> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>  		}
>>  	}
>>  
>> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
>> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
>>  	if (unconf) {
>>  		/*
>>  		 * Possible retry or client restart.  Per 18.35.4 case 4,
>> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>  	strhashval = clientstr_hashval(dname);
>>  
>>  	nfs4_lock_state();
>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
>>  	if (conf) {
>>  		/* RFC 3530 14.2.33 CASE 0: */
>>  		status = nfserr_clid_inuse;
>> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>>  	 * has a description of SETCLIENTID request processing consisting
>>  	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
>>  	 */
>> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
>> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
>>  	status = nfserr_resource;
>>  	if (!conf) {
>>  		/*
>> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>>  			unsigned int hash =
>>  				clientstr_hashval(unconf->cl_recdir);
>>  			conf = find_confirmed_client_by_str(unconf->cl_recdir,
>> -									hash);
>> +							    hash, false);
>>  			if (conf) {
>>  				nfsd4_remove_clid_dir(conf);
>>  				expire_client(conf);
>> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
>>  }
>>  
>>  int
>> -nfs4_has_reclaimed_state(const char *name)
>> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
>>  {
>>  	unsigned int strhashval = clientstr_hashval(name);
>>  	struct nfs4_client *clp;
>>  
>> -	clp = find_confirmed_client_by_str(name, strhashval);
>> +	clp = find_confirmed_client_by_str(name, strhashval, use_exchange_id);
>>  	return clp ? 1 : 0;
>>  }
>>  
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index 5de36a7..feab6ec 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char *recdir_name);
>>  extern int nfsd4_recdir_load(void);
>>  extern void nfsd4_shutdown_recdir(void);
>>  extern int nfs4_client_to_reclaim(const char *name);
>> -extern int nfs4_has_reclaimed_state(const char *name);
>> +extern int nfs4_has_reclaimed_state(const char *name, bool use_exchange_id);
>>  extern void nfsd4_recdir_purge_old(void);
>>  extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
>>  extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
>> -- 
>> 1.6.2.1
>>

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[Andy Adamson]

On Mar 31, 2009, at 4:49 AM, Benny Halevy wrote:

> On Mar. 31, 2009, 6:04 +0300, "J. Bruce Fields"  
> <bfields@fieldses.org> wrote:
>> On Sat, Mar 28, 2009 at 11:32:17AM +0300, Benny Halevy wrote:
>>> From: Andy Adamson <andros@netapp.com>
>>>
>>> We need to distinguish between client names provided by NFSv4.0  
>>> clients
>>> SETCLIENTID and those provided by NFSv4.1 via EXCHANGE_ID when  
>>> looking
>>> up the clientid by string.
>>>
>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>> [nfsd41: use boolean values for use_exchange_id argument]
>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>> ---
>>> fs/nfsd/nfs4recover.c      |    3 ++-
>>> fs/nfsd/nfs4state.c        |   41 +++++++++++++++++++++++++++++ 
>>> +-----------
>>> include/linux/nfsd/state.h |    2 +-
>>> 3 files changed, 33 insertions(+), 13 deletions(-)
>>>
>>> diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
>>> index b11cf8d..3444c00 100644
>>> --- a/fs/nfsd/nfs4recover.c
>>> +++ b/fs/nfsd/nfs4recover.c
>>> @@ -344,7 +344,8 @@ purge_old(struct dentry *parent, struct dentry  
>>> *child)
>>> {
>>> 	int status;
>>>
>>> -	if (nfs4_has_reclaimed_state(child->d_name.name))
>>> +	/* note: we currently use this path only for minorversion 0 */
>>
>> Why is that?
>
> Hmm, I'm not sure this is true anymore.
> Andy, do you recall?

I believe this was very early code intended to address the recovery of  
clientid's established via exchange_id which was never addressed.
>
> One thing for sure, we currently implemented nothing to
> propagate the "use_exchange_id" state onto the state
> recovery mechanisms, so this comment merely reflects
> that, though it isn't clear what "this path" means
> in this context, i.e. is this the path we were called
> in, or the path we're calling.

Since nfsd4_create_clid_dir is only called in nfsd4_clientid_confirm,  
we never store a clientid established by exchange_id in the recovery  
directory.
>
> At any rate, if this is something we need to fix for 4.1
> and it does not introduce any regression to 4.0, and if
> the fix isn't trivial/simple, I suggest we add a FIXME comment,
> and add it to our todo list to defer the solution post
> this push effort.

I agree. Update the above comment to a FIXME.

-->Andy

>
>
>>
>>> +	if (nfs4_has_reclaimed_state(child->d_name.name, false))
>>> 		return 0;
>>>
>>> 	status = nfsd4_clear_clid_dir(parent, child);
>>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>>> index 09c63ff..0c39376 100644
>>> --- a/fs/nfsd/nfs4state.c
>>> +++ b/fs/nfsd/nfs4state.c
>>> @@ -723,25 +723,44 @@ find_unconfirmed_client(clientid_t *clid)
>>> 	return NULL;
>>> }
>>>
>>> +/*
>>> + * Return 1 iff clp's clientid establishment method matches the  
>>> use_exchange_id
>>> + * parameter. Matching is based on the fact the at least one of the
>>> + * EXCHGID4_FLAG_USE_{NON_PNFS,PNFS_MDS,PNFS_DS} flags must be  
>>> set for v4.1
>>> + */
>>> +static inline int
>>> +match_clientid_establishment(struct nfs4_client *clp, bool  
>>> use_exchange_id)
>>> +{
>>> +#if defined(CONFIG_NFSD_V4_1)
>>> +	return (clp->cl_exchange_flags != 0) == (use_exchange_id !=  
>>> false);
>>> +#else  /* CONFIG_NFSD_V4_1 */
>>> +	return 1;
>>> +#endif /* CONFIG_NFSD_V4_1 */
>>> +}
>>
>> If the point is just to ensure that clients only match clients of the
>> same minorversion, why not just call this  
>> match_client_minorversion()?
>> You could still use cl_exchange_flags as the way to distinguish 4.0  
>> from
>> 4.1, but hide that detail away here.  In which case clearer might be:
>>
>> 	static inline u32 client_minorversion(struct nfs4_client *clp)
>> 	{
>> 		/*
>> 		 * Note 4.1 clients always have one of
>> 		 * EXCHGID4_FLAG_USE{NON_PNFS,PNFS_MDS,PNFS_DS} set.
>> 		 */
>> 		return clp->cl_exchange_flags != 0;
>> 	}
>>
>> 	static inline int client_same_minorversion(nfs4_client *clp, u32  
>> minorversion)
>> 	{
>> 		return client_minorversion(clp) == minorversion;
>> 	}
>>
>> or even just open-code the latter.
>
> I don't like using "minorversion" here since it is a numeric attribute
> and may be larger than 1 in the future.  What we care about here is
> whether the clientid was established via EXCHANGE_ID or via nfsv4.0
> SET_CLIENTID et al, therefore we used cl_exchange_flags as an  
> indication.
>
>>
>> But: are the 4.0 and 4.1 client owner-name namespaces actually  
>> meant to
>> be distinct?  2.4.1 has me a bit confused here.
>
> This case is not implemented yet (note to self - update Doc)
> If we implement it, and deal with v4.1 -> v4.1 downgrade (or  
> prevention
> thereof), then I think we can indeed unify the clientid spaces.
> However, 2.4.1 is optional and we don't have to implement it right  
> now.
>
> Benny
>
>>
>> --b.
>>
>>> +
>>> static struct nfs4_client *
>>> -find_confirmed_client_by_str(const char *dname, unsigned int  
>>> hashval)
>>> +find_confirmed_client_by_str(const char *dname, unsigned int  
>>> hashval,
>>> +			     bool use_exchange_id)
>>> {
>>> 	struct nfs4_client *clp;
>>>
>>> 	list_for_each_entry(clp, &conf_str_hashtbl[hashval], cl_strhash) {
>>> -		if (same_name(clp->cl_recdir, dname))
>>> +		if (same_name(clp->cl_recdir, dname) &&
>>> +		    match_clientid_establishment(clp, use_exchange_id))
>>> 			return clp;
>>> 	}
>>> 	return NULL;
>>> }
>>>
>>> static struct nfs4_client *
>>> -find_unconfirmed_client_by_str(const char *dname, unsigned int  
>>> hashval)
>>> +find_unconfirmed_client_by_str(const char *dname, unsigned int  
>>> hashval,
>>> +			       bool use_exchange_id)
>>> {
>>> 	struct nfs4_client *clp;
>>>
>>> 	list_for_each_entry(clp, &unconf_str_hashtbl[hashval],  
>>> cl_strhash) {
>>> -		if (same_name(clp->cl_recdir, dname))
>>> +		if (same_name(clp->cl_recdir, dname) &&
>>> +		    match_clientid_establishment(clp, use_exchange_id))
>>> 			return clp;
>>> 	}
>>> 	return NULL;
>>> @@ -895,7 +914,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>> 	nfs4_lock_state();
>>> 	status = nfs_ok;
>>>
>>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>>> +	conf = find_confirmed_client_by_str(dname, strhashval, true);
>>> 	if (conf) {
>>> 		if (!same_verf(&verf, &conf->cl_verifier)) {
>>> 			/* 18.35.4 case 8 */
>>> @@ -943,7 +962,7 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
>>> 		}
>>> 	}
>>>
>>> -	unconf  = find_unconfirmed_client_by_str(dname, strhashval);
>>> +	unconf  = find_unconfirmed_client_by_str(dname, strhashval, true);
>>> 	if (unconf) {
>>> 		/*
>>> 		 * Possible retry or client restart.  Per 18.35.4 case 4,
>>> @@ -1041,7 +1060,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp,  
>>> struct nfsd4_compound_state *cstate,
>>> 	strhashval = clientstr_hashval(dname);
>>>
>>> 	nfs4_lock_state();
>>> -	conf = find_confirmed_client_by_str(dname, strhashval);
>>> +	conf = find_confirmed_client_by_str(dname, strhashval, false);
>>> 	if (conf) {
>>> 		/* RFC 3530 14.2.33 CASE 0: */
>>> 		status = nfserr_clid_inuse;
>>> @@ -1056,7 +1075,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp,  
>>> struct nfsd4_compound_state *cstate,
>>> 	 * has a description of SETCLIENTID request processing consisting
>>> 	 * of 5 bullet points, labeled as CASE0 - CASE4 below.
>>> 	 */
>>> -	unconf = find_unconfirmed_client_by_str(dname, strhashval);
>>> +	unconf = find_unconfirmed_client_by_str(dname, strhashval, false);
>>> 	status = nfserr_resource;
>>> 	if (!conf) {
>>> 		/*
>>> @@ -1211,7 +1230,7 @@ nfsd4_setclientid_confirm(struct svc_rqst  
>>> *rqstp,
>>> 			unsigned int hash =
>>> 				clientstr_hashval(unconf->cl_recdir);
>>> 			conf = find_confirmed_client_by_str(unconf->cl_recdir,
>>> -									hash);
>>> +							    hash, false);
>>> 			if (conf) {
>>> 				nfsd4_remove_clid_dir(conf);
>>> 				expire_client(conf);
>>> @@ -3332,12 +3351,12 @@ alloc_reclaim(void)
>>> }
>>>
>>> int
>>> -nfs4_has_reclaimed_state(const char *name)
>>> +nfs4_has_reclaimed_state(const char *name, bool use_exchange_id)
>>> {
>>> 	unsigned int strhashval = clientstr_hashval(name);
>>> 	struct nfs4_client *clp;
>>>
>>> -	clp = find_confirmed_client_by_str(name, strhashval);
>>> +	clp = find_confirmed_client_by_str(name, strhashval,  
>>> use_exchange_id);
>>> 	return clp ? 1 : 0;
>>> }
>>>
>>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>>> index 5de36a7..feab6ec 100644
>>> --- a/include/linux/nfsd/state.h
>>> +++ b/include/linux/nfsd/state.h
>>> @@ -331,7 +331,7 @@ extern void nfsd4_init_recdir(char  
>>> *recdir_name);
>>> extern int nfsd4_recdir_load(void);
>>> extern void nfsd4_shutdown_recdir(void);
>>> extern int nfs4_client_to_reclaim(const char *name);
>>> -extern int nfs4_has_reclaimed_state(const char *name);
>>> +extern int nfs4_has_reclaimed_state(const char *name, bool  
>>> use_exchange_id);
>>> extern void nfsd4_recdir_purge_old(void);
>>> extern int nfsd4_create_clid_dir(struct nfs4_client *clp);
>>> extern void nfsd4_remove_clid_dir(struct nfs4_client *clp);
>>> -- 
>>> 1.6.2.1
>>>

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[J. Bruce Fields]

On Tue, Mar 31, 2009 at 10:10:48AM -0400, Andy Adamson wrote:
> On Mar 31, 2009, at 4:49 AM, Benny Halevy wrote:
>> At any rate, if this is something we need to fix for 4.1
>> and it does not introduce any regression to 4.0, and if
>> the fix isn't trivial/simple, I suggest we add a FIXME comment,
>> and add it to our todo list to defer the solution post
>> this push effort.

OK, apologies, it's just takes me much too long to catch up with all of
you, and make sure I understand these patches.

And I'm conflicted.

On the one hand, submission-time gives a really clear point at which to
do review and handle any problems found.  I'm a little worried that some
problems will be forgotten once the code is in.

On the other hand, there's a lot of 4.1 development going on and it
would be better to see it happening in mainline than out.  I don't see
any more v2/v3/v4.0 regressions, and people in general seem willing to
track and respond to comments.

On the other other hand, I do at least want to reassure myself that this
is a reasonable basis for further development.

I'd like to put off the callback stuff for now, at least: the addition
of a mutex held over the callbacks worries me, and interferes with an
ongoing attempt to make all the callback code asynchronous.  And I
haven't tried to review the rpc-level code there yet (and haven't seen
review of it from Trond).

Would it be possible to get one last revison of the patch series which
stops short of the callback stuff (so, the first 32 patches or so), and
fixes the trivial-to-fix stuff?  And would it be possible to do that by
(ulp) tomorrow?

--b.

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[J. Bruce Fields]

By the way, one more random question: the basic tests I run before
submitting patches, for example, are mainly just pynfs and cthon.  I'd
should start adding some 4.1 tests: suggestions?  E.g. could anyone
suggest a reasonable commandline for the 4.1 pynfs tests that excludes
known failing tests?

--b.

Re: [PATCH v2 16/47] nfsd41: match clientid establishment method

[Benny Halevy]

On Apr. 02, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Tue, Mar 31, 2009 at 10:10:48AM -0400, Andy Adamson wrote:
>> On Mar 31, 2009, at 4:49 AM, Benny Halevy wrote:
>>> At any rate, if this is something we need to fix for 4.1
>>> and it does not introduce any regression to 4.0, and if
>>> the fix isn't trivial/simple, I suggest we add a FIXME comment,
>>> and add it to our todo list to defer the solution post
>>> this push effort.
> 
> OK, apologies, it's just takes me much too long to catch up with all of
> you, and make sure I understand these patches.
> 
> And I'm conflicted.
> 
> On the one hand, submission-time gives a really clear point at which to
> do review and handle any problems found.  I'm a little worried that some
> problems will be forgotten once the code is in.
> 
> On the other hand, there's a lot of 4.1 development going on and it
> would be better to see it happening in mainline than out.  I don't see
> any more v2/v3/v4.0 regressions, and people in general seem willing to
> track and respond to comments.
> 
> On the other other hand, I do at least want to reassure myself that this
> is a reasonable basis for further development.

Absolutely.  I was just trying to set the focus on the top priority, IMO:
flushing out the interaction with v4.0 and making sure there are no
regressions. I know that the v4.1 code is far from being perfect, even
with respect to the spec and I like your idea of at least documenting what
we have and what we don't, and what needs fixing.

As you said, keeping the code out of tree doesn't make it any
easier to fix or maintain so getting the new code in, in spite
of possible flaws with the *new* functionality as long as it's in a good
enough shape as a starting point, can just accelerate its development.

> 
> I'd like to put off the callback stuff for now, at least: the addition
> of a mutex held over the callbacks worries me, and interferes with an
> ongoing attempt to make all the callback code asynchronous.  And I
> haven't tried to review the rpc-level code there yet (and haven't seen
> review of it from Trond).

Understood.

> 
> Would it be possible to get one last revison of the patch series which
> stops short of the callback stuff (so, the first 32 patches or so), and
> fixes the trivial-to-fix stuff?  And would it be possible to do that by
> (ulp) tomorrow?

I'm on it.

Benny

> 
> --b.

[PATCH v2 17/47] nfsd41: sequence operation

[Benny Halevy]

Implement the sequence operation conforming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

Check for stale clientid (as derived from the sessionid).
Enforce slotid range and exactly-once semantics using
the slotid and seqid.

If everything went well renew the client lease and
mark the slot INPROGRESS.

[nfsd41: rename sequence catchthis to cachethis]
Signed-off-by: Andy Adamson<andros@netapp.com>
[pulled some code to set cstate->slot from "nfsd DRC logic"]
[use sessionid_lock spin lock]
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c       |   71 +++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c         |   32 +++++++++++++++++++-
 include/linux/nfsd/xdr4.h |   10 ++++++-
 3 files changed, 108 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 0c39376..a19f292 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1004,6 +1004,32 @@ error:
 	return status;
 }
 
+static int
+check_slot_seqid(u32 seqid, struct nfsd4_slot *slot)
+{
+	dprintk("%s enter. seqid %d slot->sl_seqid %d\n", __func__, seqid,
+		slot->sl_seqid);
+
+	/* The slot is in use, and no response has been sent. */
+	if (slot->sl_inuse) {
+		if (seqid == slot->sl_seqid)
+			return nfserr_jukebox;
+		else
+			return nfserr_seq_misordered;
+	}
+	/* Normal */
+	if (likely(seqid == slot->sl_seqid + 1))
+		return nfs_ok;
+	/* Replay */
+	if (seqid == slot->sl_seqid)
+		return nfserr_replay_cache;
+	/* Wraparound */
+	if (seqid == 1 && (slot->sl_seqid + 1) == 0)
+		return nfs_ok;
+	/* Misordered replay or misordered new request */
+	return nfserr_seq_misordered;
+}
+
 __be32
 nfsd4_create_session(struct svc_rqst *rqstp,
 		     struct nfsd4_compound_state *cstate,
@@ -1021,11 +1047,52 @@ nfsd4_destroy_session(struct svc_rqst *r,
 }
 
 __be32
-nfsd4_sequence(struct svc_rqst *r,
+nfsd4_sequence(struct svc_rqst *rqstp,
 	       struct nfsd4_compound_state *cstate,
 	       struct nfsd4_sequence *seq)
 {
-	return -1;	/* stub */
+	struct nfsd4_session *session;
+	struct nfsd4_slot *slot;
+	int status;
+
+	spin_lock(&sessionid_lock);
+	status = nfserr_badsession;
+	session = find_in_sessionid_hashtbl(&seq->sessionid);
+	if (!session)
+		goto out;
+
+	status = nfserr_badslot;
+	if (seq->slotid >= session->se_fnumslots)
+		goto out;
+
+	slot = &session->se_slots[seq->slotid];
+	dprintk("%s: slotid %d\n", __func__, seq->slotid);
+
+	status = check_slot_seqid(seq->seqid, slot);
+	if (status == nfserr_replay_cache) {
+		cstate->slot = slot;
+		goto replay_cache;
+	}
+	if (status)
+		goto out;
+
+	/* Success! bump slot seqid */
+	slot->sl_inuse = true;
+	slot->sl_seqid = seq->seqid;
+
+	cstate->slot = slot;
+
+replay_cache:
+	/* Renew the clientid on success and on replay.
+	 * Hold a session reference until done processing the compound:
+	 * nfsd4_put_session called only if the cstate slot is set.
+	 */
+	renew_client(session->se_client);
+	nfsd4_get_session(slot->sl_session);
+out:
+	spin_unlock(&sessionid_lock);
+	dprintk("%s: return %d\n", __func__, ntohl(status));
+	return status;
 }
 #endif /* CONFIG_NFSD_V4_1 */
 
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 840cf6a..c6b490e 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1114,7 +1114,16 @@ static __be32
 nfsd4_decode_sequence(struct nfsd4_compoundargs *argp,
 		      struct nfsd4_sequence *seq)
 {
-	return nfserr_opnotsupp;	/* stub */
+	DECODE_HEAD;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	COPYMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	READ32(seq->seqid);
+	READ32(seq->slotid);
+	READ32(seq->maxslots);
+	READ32(seq->cachethis);
+
+	DECODE_TAIL;
 }
 #endif /* CONFIG_NFSD_V4_1 */
 
@@ -2836,7 +2845,26 @@ static __be32
 nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
 		      struct nfsd4_sequence *seq)
 {
-	/* stub */
+	ENCODE_HEAD;
+
+	if (nfserr)
+		goto out;
+
+	RESERVE_SPACE(NFS4_MAX_SESSIONID_LEN + 20);
+	WRITEMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(seq->seqid);
+	WRITE32(seq->slotid);
+	WRITE32(seq->maxslots);
+	/*
+	 * FIXME: for now:
+	 *   target_maxslots = maxslots
+	 *   status_flags = 0
+	 */
+	WRITE32(seq->maxslots);
+	WRITE32(0);
+
+	ADJUST_ARGS();
+out:
 	return nfserr;
 }
 #endif /* CONFIG_NFSD_V4_1 */
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index ea5a427..9e4d8db 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -362,7 +362,15 @@ struct nfsd4_create_session {
 };
 
 struct nfsd4_sequence {
-	int	foo;	/* stub */
+	struct nfs4_sessionid	sessionid;		/* request/response */
+	u32			seqid;			/* request/response */
+	u32			slotid;			/* request/response */
+	u32			maxslots;		/* request/response */
+	u32			cachethis;		/* request */
+#if 0
+	u32			target_maxslots;	/* response */
+	u32			status_flags;		/* response */
+#endif /* not yet */
 };
 
 struct nfsd4_destroy_session {
-- 
1.6.2.1

Re: [PATCH v2 17/47] nfsd41: sequence operation

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:21AM +0300, Benny Halevy wrote:
> Implement the sequence operation conforming to
> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26
> 
> Check for stale clientid (as derived from the sessionid).
> Enforce slotid range and exactly-once semantics using
> the slotid and seqid.
> 
> If everything went well renew the client lease and
> mark the slot INPROGRESS.
> 
> [nfsd41: rename sequence catchthis to cachethis]
> Signed-off-by: Andy Adamson<andros@netapp.com>
> [pulled some code to set cstate->slot from "nfsd DRC logic"]
> [use sessionid_lock spin lock]
> [nfsd41: use bool inuse for slot state]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c       |   71 +++++++++++++++++++++++++++++++++++++++++++-
>  fs/nfsd/nfs4xdr.c         |   32 +++++++++++++++++++-
>  include/linux/nfsd/xdr4.h |   10 ++++++-
>  3 files changed, 108 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 0c39376..a19f292 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -1004,6 +1004,32 @@ error:
>  	return status;
>  }
>  
> +static int
> +check_slot_seqid(u32 seqid, struct nfsd4_slot *slot)
> +{
> +	dprintk("%s enter. seqid %d slot->sl_seqid %d\n", __func__, seqid,
> +		slot->sl_seqid);
> +
> +	/* The slot is in use, and no response has been sent. */
> +	if (slot->sl_inuse) {
> +		if (seqid == slot->sl_seqid)
> +			return nfserr_jukebox;
> +		else
> +			return nfserr_seq_misordered;
> +	}
> +	/* Normal */
> +	if (likely(seqid == slot->sl_seqid + 1))
> +		return nfs_ok;
> +	/* Replay */
> +	if (seqid == slot->sl_seqid)
> +		return nfserr_replay_cache;
> +	/* Wraparound */
> +	if (seqid == 1 && (slot->sl_seqid + 1) == 0)
> +		return nfs_ok;
> +	/* Misordered replay or misordered new request */
> +	return nfserr_seq_misordered;
> +}
> +
>  __be32
>  nfsd4_create_session(struct svc_rqst *rqstp,
>  		     struct nfsd4_compound_state *cstate,
> @@ -1021,11 +1047,52 @@ nfsd4_destroy_session(struct svc_rqst *r,
>  }
>  
>  __be32
> -nfsd4_sequence(struct svc_rqst *r,
> +nfsd4_sequence(struct svc_rqst *rqstp,
>  	       struct nfsd4_compound_state *cstate,
>  	       struct nfsd4_sequence *seq)
>  {
> -	return -1;	/* stub */
> +	struct nfsd4_session *session;
> +	struct nfsd4_slot *slot;
> +	int status;
> +
> +	spin_lock(&sessionid_lock);
> +	status = nfserr_badsession;
> +	session = find_in_sessionid_hashtbl(&seq->sessionid);
> +	if (!session)
> +		goto out;
> +
> +	status = nfserr_badslot;
> +	if (seq->slotid >= session->se_fnumslots)
> +		goto out;
> +
> +	slot = &session->se_slots[seq->slotid];
> +	dprintk("%s: slotid %d\n", __func__, seq->slotid);
> +
> +	status = check_slot_seqid(seq->seqid, slot);
> +	if (status == nfserr_replay_cache) {
> +		cstate->slot = slot;
> +		goto replay_cache;
> +	}
> +	if (status)
> +		goto out;
> +
> +	/* Success! bump slot seqid */
> +	slot->sl_inuse = true;
> +	slot->sl_seqid = seq->seqid;
> +
> +	cstate->slot = slot;
> +
> +replay_cache:
> +	/* Renew the clientid on success and on replay.
> +	 * Hold a session reference until done processing the compound:
> +	 * nfsd4_put_session called only if the cstate slot is set.
> +	 */
> +	renew_client(session->se_client);
> +	nfsd4_get_session(slot->sl_session);
> +out:
> +	spin_unlock(&sessionid_lock);
> +	dprintk("%s: return %d\n", __func__, ntohl(status));
> +	return status;
>  }
>  #endif /* CONFIG_NFSD_V4_1 */
>  
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index 840cf6a..c6b490e 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -1114,7 +1114,16 @@ static __be32
>  nfsd4_decode_sequence(struct nfsd4_compoundargs *argp,
>  		      struct nfsd4_sequence *seq)
>  {
> -	return nfserr_opnotsupp;	/* stub */
> +	DECODE_HEAD;
> +
> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
> +	COPYMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
> +	READ32(seq->seqid);
> +	READ32(seq->slotid);
> +	READ32(seq->maxslots);
> +	READ32(seq->cachethis);
> +
> +	DECODE_TAIL;
>  }
>  #endif /* CONFIG_NFSD_V4_1 */
>  
> @@ -2836,7 +2845,26 @@ static __be32
>  nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
>  		      struct nfsd4_sequence *seq)
>  {
> -	/* stub */
> +	ENCODE_HEAD;
> +
> +	if (nfserr)
> +		goto out;

Just 'return nfserr'.  I don't see the point of the goto if it's
literally just replacing a return.

> +
> +	RESERVE_SPACE(NFS4_MAX_SESSIONID_LEN + 20);
> +	WRITEMEM(seq->sessionid.data, NFS4_MAX_SESSIONID_LEN);
> +	WRITE32(seq->seqid);
> +	WRITE32(seq->slotid);
> +	WRITE32(seq->maxslots);
> +	/*
> +	 * FIXME: for now:
> +	 *   target_maxslots = maxslots
> +	 *   status_flags = 0
> +	 */
> +	WRITE32(seq->maxslots);
> +	WRITE32(0);
> +
> +	ADJUST_ARGS();
> +out:
>  	return nfserr;
>  }
>  #endif /* CONFIG_NFSD_V4_1 */
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index ea5a427..9e4d8db 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -362,7 +362,15 @@ struct nfsd4_create_session {
>  };
>  
>  struct nfsd4_sequence {
> -	int	foo;	/* stub */
> +	struct nfs4_sessionid	sessionid;		/* request/response */
> +	u32			seqid;			/* request/response */
> +	u32			slotid;			/* request/response */
> +	u32			maxslots;		/* request/response */
> +	u32			cachethis;		/* request */
> +#if 0
> +	u32			target_maxslots;	/* response */
> +	u32			status_flags;		/* response */
> +#endif /* not yet */

I'd rather that be patched in when it's needed, but OK.

--b.

>  };
>  
>  struct nfsd4_destroy_session {
> -- 
> 1.6.2.1
> 

[PATCH v2 18/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c  |   24 ++++++++++++++++--------
 fs/nfsd/nfs4state.c |    4 ++++
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f618e8e..e703ac2 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -811,14 +811,15 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
 
 typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
 			      void *);
+enum nfsd4_op_flags {
+	ALLOWED_WITHOUT_FH = 1 << 0,	/* No current filehandle required */
+	ALLOWED_ON_ABSENT_FS = 2 << 0,	/* ops processed on absent fs */
+	ALLOWED_AS_FIRST_OP = 3 << 0,	/* ops reqired first in compound */
+};
 
 struct nfsd4_operation {
 	nfsd4op_func op_func;
 	u32 op_flags;
-/* Most ops require a valid current filehandle; a few don't: */
-#define ALLOWED_WITHOUT_FH 1
-/* GETATTR and ops not listed as returning NFS4ERR_MOVED: */
-#define ALLOWED_ON_ABSENT_FS 2
 	char *op_name;
 };
 
@@ -864,6 +865,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
 		goto out;
 
+	op = &args->ops[0];
+	if (args->opcnt > 0 && op->status != nfserr_op_illegal &&
+	    !(nfsd4_ops[op->opnum].op_flags & ALLOWED_AS_FIRST_OP)) {
+		op->status = nfserr_sequence_pos;
+		goto encode_op;
+	}
+
 	status = nfs_ok;
 	while (!status && resp->opcnt < args->opcnt) {
 		op = &args->ops[resp->opcnt++];
@@ -1104,22 +1112,22 @@ static struct nfsd4_operation nfsd4_ops[] = {
 #if defined(CONFIG_NFSD_V4_1)
 	[OP_EXCHANGE_ID] = {
 		.op_func = (nfsd4op_func)nfsd4_exchange_id,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_EXCHANGE_ID",
 	},
 	[OP_CREATE_SESSION] = {
 		.op_func = (nfsd4op_func)nfsd4_create_session,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_CREATE_SESSION",
 	},
 	[OP_DESTROY_SESSION] = {
 		.op_func = (nfsd4op_func)nfsd4_destroy_session,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_DESTROY_SESSION",
 	},
 	[OP_SEQUENCE] = {
 		.op_func = (nfsd4op_func)nfsd4_sequence,
-		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
 		.op_name = "OP_SEQUENCE",
 	},
 #endif /* CONFIG_NFSD_V4_1 */
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index a19f292..10eb67b 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1051,10 +1051,14 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	       struct nfsd4_compound_state *cstate,
 	       struct nfsd4_sequence *seq)
 {
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfsd4_session *session;
 	struct nfsd4_slot *slot;
 	int status;
 
+	if (resp->opcnt != 1)
+		return nfserr_sequence_pos;
+
 	spin_lock(&sessionid_lock);
 	status = nfserr_badsession;
 	session = find_in_sessionid_hashtbl(&seq->sessionid);
-- 
1.6.2.1

Re: [PATCH v2 18/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules

[J. Bruce Fields]

There are a number of other ordering requirements too:

	- EXCHANGE_ID must be the only op if there's no sequence
	- DESTROY_SESSION must be the last op in its compound, I think?

Are there others?  And are these enforced somewhere as well?

--b.

On Sat, Mar 28, 2009 at 11:32:29AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Signed-off-by: Andy Adamson<andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4proc.c  |   24 ++++++++++++++++--------
>  fs/nfsd/nfs4state.c |    4 ++++
>  2 files changed, 20 insertions(+), 8 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index f618e8e..e703ac2 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -811,14 +811,15 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
>  
>  typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
>  			      void *);
> +enum nfsd4_op_flags {
> +	ALLOWED_WITHOUT_FH = 1 << 0,	/* No current filehandle required */
> +	ALLOWED_ON_ABSENT_FS = 2 << 0,	/* ops processed on absent fs */
> +	ALLOWED_AS_FIRST_OP = 3 << 0,	/* ops reqired first in compound */
> +};
>  
>  struct nfsd4_operation {
>  	nfsd4op_func op_func;
>  	u32 op_flags;
> -/* Most ops require a valid current filehandle; a few don't: */
> -#define ALLOWED_WITHOUT_FH 1
> -/* GETATTR and ops not listed as returning NFS4ERR_MOVED: */
> -#define ALLOWED_ON_ABSENT_FS 2
>  	char *op_name;
>  };
>  
> @@ -864,6 +865,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
>  		goto out;
>  
> +	op = &args->ops[0];
> +	if (args->opcnt > 0 && op->status != nfserr_op_illegal &&
> +	    !(nfsd4_ops[op->opnum].op_flags & ALLOWED_AS_FIRST_OP)) {
> +		op->status = nfserr_sequence_pos;
> +		goto encode_op;
> +	}
> +
>  	status = nfs_ok;
>  	while (!status && resp->opcnt < args->opcnt) {
>  		op = &args->ops[resp->opcnt++];
> @@ -1104,22 +1112,22 @@ static struct nfsd4_operation nfsd4_ops[] = {
>  #if defined(CONFIG_NFSD_V4_1)
>  	[OP_EXCHANGE_ID] = {
>  		.op_func = (nfsd4op_func)nfsd4_exchange_id,
> -		.op_flags = ALLOWED_WITHOUT_FH,
> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>  		.op_name = "OP_EXCHANGE_ID",
>  	},
>  	[OP_CREATE_SESSION] = {
>  		.op_func = (nfsd4op_func)nfsd4_create_session,
> -		.op_flags = ALLOWED_WITHOUT_FH,
> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>  		.op_name = "OP_CREATE_SESSION",
>  	},
>  	[OP_DESTROY_SESSION] = {
>  		.op_func = (nfsd4op_func)nfsd4_destroy_session,
> -		.op_flags = ALLOWED_WITHOUT_FH,
> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>  		.op_name = "OP_DESTROY_SESSION",
>  	},
>  	[OP_SEQUENCE] = {
>  		.op_func = (nfsd4op_func)nfsd4_sequence,
> -		.op_flags = ALLOWED_WITHOUT_FH,
> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>  		.op_name = "OP_SEQUENCE",
>  	},
>  #endif /* CONFIG_NFSD_V4_1 */
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index a19f292..10eb67b 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -1051,10 +1051,14 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>  	       struct nfsd4_compound_state *cstate,
>  	       struct nfsd4_sequence *seq)
>  {
> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>  	struct nfsd4_session *session;
>  	struct nfsd4_slot *slot;
>  	int status;
>  
> +	if (resp->opcnt != 1)
> +		return nfserr_sequence_pos;
> +
>  	spin_lock(&sessionid_lock);
>  	status = nfserr_badsession;
>  	session = find_in_sessionid_hashtbl(&seq->sessionid);
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 18/47] nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules

[Benny Halevy]

On Mar. 31, 2009, 6:20 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> There are a number of other ordering requirements too:
> 
> 	- EXCHANGE_ID must be the only op if there's no sequence
> 	- DESTROY_SESSION must be the last op in its compound, I think?

Right on the spot (with one reservation).

   If the COMPOUND request starts with SEQUENCE, and if the sessionids
   specified in SEQUENCE and DESTROY_SESSION are the same, then

   o  DESTROY_SESSION MUST be the final operation in the COMPOUND
      request.

Though I'm not sure what error should the server return in this case,
NFS4ERR_BADXDR maybe?  But even if we don't enforce it, any op referring
to the destroyed session coming after DESTROY_SESSION must fail on
NFS4ERR_BADSESSION.  An interesting case will be CREATE_SESSION
following DESTROY_SESSION in a COMPOUND starting with SEQUENCE... :-)

> 
> Are there others?  And are these enforced somewhere as well?

Good catch!  These aren't enforced yet (note to self - Doc).
Other than that, I see:
   | NFS4ERR_NOT_ONLY_OP               | BIND_CONN_TO_SESSION,         |
   |                                   | CREATE_SESSION,               |
   |                                   | DESTROY_CLIENTID,             |
   |                                   | DESTROY_SESSION, EXCHANGE_ID  |
- If CREATE_SESSION is sent without a
   preceding SEQUENCE, then it MUST be the only operation in the
   COMPOUND procedure's request.

Benny

> 
> --b.
> 
> On Sat, Mar 28, 2009 at 11:32:29AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Signed-off-by: Andy Adamson<andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4proc.c  |   24 ++++++++++++++++--------
>>  fs/nfsd/nfs4state.c |    4 ++++
>>  2 files changed, 20 insertions(+), 8 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>> index f618e8e..e703ac2 100644
>> --- a/fs/nfsd/nfs4proc.c
>> +++ b/fs/nfsd/nfs4proc.c
>> @@ -811,14 +811,15 @@ static inline void nfsd4_increment_op_stats(u32 opnum)
>>  
>>  typedef __be32(*nfsd4op_func)(struct svc_rqst *, struct nfsd4_compound_state *,
>>  			      void *);
>> +enum nfsd4_op_flags {
>> +	ALLOWED_WITHOUT_FH = 1 << 0,	/* No current filehandle required */
>> +	ALLOWED_ON_ABSENT_FS = 2 << 0,	/* ops processed on absent fs */
>> +	ALLOWED_AS_FIRST_OP = 3 << 0,	/* ops reqired first in compound */
>> +};
>>  
>>  struct nfsd4_operation {
>>  	nfsd4op_func op_func;
>>  	u32 op_flags;
>> -/* Most ops require a valid current filehandle; a few don't: */
>> -#define ALLOWED_WITHOUT_FH 1
>> -/* GETATTR and ops not listed as returning NFS4ERR_MOVED: */
>> -#define ALLOWED_ON_ABSENT_FS 2
>>  	char *op_name;
>>  };
>>  
>> @@ -864,6 +865,13 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>>  	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
>>  		goto out;
>>  
>> +	op = &args->ops[0];
>> +	if (args->opcnt > 0 && op->status != nfserr_op_illegal &&
>> +	    !(nfsd4_ops[op->opnum].op_flags & ALLOWED_AS_FIRST_OP)) {
>> +		op->status = nfserr_sequence_pos;
>> +		goto encode_op;
>> +	}
>> +
>>  	status = nfs_ok;
>>  	while (!status && resp->opcnt < args->opcnt) {
>>  		op = &args->ops[resp->opcnt++];
>> @@ -1104,22 +1112,22 @@ static struct nfsd4_operation nfsd4_ops[] = {
>>  #if defined(CONFIG_NFSD_V4_1)
>>  	[OP_EXCHANGE_ID] = {
>>  		.op_func = (nfsd4op_func)nfsd4_exchange_id,
>> -		.op_flags = ALLOWED_WITHOUT_FH,
>> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>>  		.op_name = "OP_EXCHANGE_ID",
>>  	},
>>  	[OP_CREATE_SESSION] = {
>>  		.op_func = (nfsd4op_func)nfsd4_create_session,
>> -		.op_flags = ALLOWED_WITHOUT_FH,
>> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>>  		.op_name = "OP_CREATE_SESSION",
>>  	},
>>  	[OP_DESTROY_SESSION] = {
>>  		.op_func = (nfsd4op_func)nfsd4_destroy_session,
>> -		.op_flags = ALLOWED_WITHOUT_FH,
>> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>>  		.op_name = "OP_DESTROY_SESSION",
>>  	},
>>  	[OP_SEQUENCE] = {
>>  		.op_func = (nfsd4op_func)nfsd4_sequence,
>> -		.op_flags = ALLOWED_WITHOUT_FH,
>> +		.op_flags = ALLOWED_WITHOUT_FH | ALLOWED_AS_FIRST_OP,
>>  		.op_name = "OP_SEQUENCE",
>>  	},
>>  #endif /* CONFIG_NFSD_V4_1 */
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index a19f292..10eb67b 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -1051,10 +1051,14 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>>  	       struct nfsd4_compound_state *cstate,
>>  	       struct nfsd4_sequence *seq)
>>  {
>> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>>  	struct nfsd4_session *session;
>>  	struct nfsd4_slot *slot;
>>  	int status;
>>  
>> +	if (resp->opcnt != 1)
>> +		return nfserr_sequence_pos;
>> +
>>  	spin_lock(&sessionid_lock);
>>  	status = nfserr_badsession;
>>  	session = find_in_sessionid_hashtbl(&seq->sessionid);
>> -- 
>> 1.6.2.1
>>

[PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Cache all the result pages, including the rpc header in rq_respages[0],
for a request in the slot table cache entry.

Cache the statp pointer from nfsd_dispatch which points into rq_respages[0]
just past the rpc header. When setting a cache entry, calculate and save the
length of the nfs data minus the rpc header for rq_respages[0].

When replaying a cache entry, replace the cached rpc header with the
replayed request rpc result header, unless there is not enough room in the
cached results first page. In that case, use the cached rpc header.

The sessions fore channel maxresponse size cached is set to NFSD_PAGES_PER_SLOT
* PAGE_SIZE. For compounds we are cacheing with operations such as READDIR
that use the xdr_buf->pages to hold data, we choose to cache the extra page of
data rather than copying data from xdr_buf->pages into the xdr_buf->head page.

[nfsd41: limit cache to maxresponsesize_cached]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfssvc.c           |    4 +
 include/linux/nfsd/cache.h |    5 ++
 include/linux/nfsd/state.h |   13 ++++
 include/linux/nfsd/xdr4.h  |    4 +
 5 files changed, 168 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 10eb67b..f0ce639 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -860,6 +860,148 @@ out_err:
 }
 
 #if defined(CONFIG_NFSD_V4_1)
+void
+nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
+{
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
+
+	resp->cstate.statp = statp;
+}
+
+/*
+ * Dereference the result pages.
+ */
+static void
+nfsd4_release_respages(struct page **respages, short resused)
+{
+	int page_no;
+
+	dprintk("--> %s\n", __func__);
+	for (page_no = 0; page_no < resused; page_no++) {
+		if (!respages[page_no])
+			continue;
+		put_page(respages[page_no]);
+		respages[page_no] = NULL;
+	}
+}
+
+static void
+nfsd4_move_pages(struct page **topages, struct page **frompages, short count)
+{
+	int page_no;
+
+	for (page_no = 0; page_no < count; page_no++) {
+		topages[page_no] = frompages[page_no];
+		if (!topages[page_no])
+			continue;
+		get_page(topages[page_no]);
+	}
+}
+
+/*
+ * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing the previous
+ * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where the total
+ * length of the XDR response is less than se_fmaxresp_cached
+ * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used for a
+ * of the reply (e.g. readdir).
+ *
+ * Store the base and length of the rq_req.head[0] page
+ * of the NFSv4.1 data, just past the rpc header.
+ */
+void
+nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
+	struct svc_rqst *rqstp = resp->rqstp;
+	struct kvec *resv = &rqstp->rq_res.head[0];
+
+	dprintk("--> %s entry %p\n", __func__, entry);
+
+	/* Don't cache a failed OP_SEQUENCE */
+	if (resp->opcnt == 1 && resp->cstate.status)
+		return;
+	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
+	entry->ce_resused = rqstp->rq_resused;
+	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
+		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
+	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
+			 entry->ce_resused);
+	entry->ce_status = resp->cstate.status;
+	entry->ce_datav.iov_base = resp->cstate.statp;
+	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
+				(char *)page_address(rqstp->rq_respages[0]));
+	entry->ce_opcnt = resp->opcnt;
+	/* Current request rpc header length*/
+	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
+				(char *)page_address(rqstp->rq_respages[0]);
+}
+
+/*
+ * Copy the cached NFSv4.1 reply skipping the cached rpc header into the
+ * replay result res.head[0] past the rpc header to end up with replay
+ * rpc header and cached NFSv4.1 reply.
+ */
+static int
+nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
+			struct nfsd4_cache_entry *entry)
+{
+	struct svc_rqst *rqstp = resp->rqstp;
+	struct kvec *resv = &resp->rqstp->rq_res.head[0];
+	int len;
+
+	/* Current request rpc header length*/
+	len = (char *)resp->cstate.statp -
+			(char *)page_address(rqstp->rq_respages[0]);
+	if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
+		dprintk("%s v41 cached reply too large (%Zd).\n", __func__,
+			entry->ce_datav.iov_len);
+		return 0;
+	}
+	/* copy the cached reply nfsd data past the current rpc header */
+	memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_base,
+		entry->ce_datav.iov_len);
+	resv->iov_len = len + entry->ce_datav.iov_len;
+	return 1;
+}
+
+/*
+ * Keep the first page of the replay. Copy the NFSv4.1 data from the first
+ * cached page.  Replace any futher replay pages from the cache.
+ */
+__be32
+nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
+	__be32 status;
+
+	dprintk("--> %s entry %p\n", __func__, entry);
+
+
+	if (!nfsd41_copy_replay_data(resp, entry)) {
+		/*
+		 * Not enough room to use the replay rpc header, send the
+		 * cached header. Release all the allocated result pages.
+		 */
+		svc_free_res_pages(resp->rqstp);
+		nfsd4_move_pages(resp->rqstp->rq_respages, entry->ce_respages,
+			entry->ce_resused);
+	} else {
+		/* Release all but the first allocated result page */
+
+		resp->rqstp->rq_resused--;
+		svc_free_res_pages(resp->rqstp);
+
+		nfsd4_move_pages(&resp->rqstp->rq_respages[1],
+				 &entry->ce_respages[1],
+				 entry->ce_resused - 1);
+	}
+
+	resp->rqstp->rq_resused = entry->ce_resused;
+	status = entry->ce_status;
+
+	return status;
+}
+
 /*
  * Set the exchange_id flags returned by the server.
  */
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index ef0a368..b5168d1 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -515,6 +515,10 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 *statp)
 		+ rqstp->rq_res.head[0].iov_len;
 	rqstp->rq_res.head[0].iov_len += sizeof(__be32);
 
+	/* NFSv4.1 DRC requires statp */
+	if (rqstp->rq_vers == 4)
+		nfsd4_set_statp(rqstp, statp);
+
 	/* Now call the procedure handler, and encode NFS status. */
 	nfserr = proc->pc_func(rqstp, rqstp->rq_argp, rqstp->rq_resp);
 	nfserr = map_new_errors(rqstp->rq_vers, nfserr);
diff --git a/include/linux/nfsd/cache.h b/include/linux/nfsd/cache.h
index 04b355c..57a83c7 100644
--- a/include/linux/nfsd/cache.h
+++ b/include/linux/nfsd/cache.h
@@ -75,5 +75,10 @@ int	nfsd_reply_cache_init(void);
 void	nfsd_reply_cache_shutdown(void);
 int	nfsd_cache_lookup(struct svc_rqst *, int);
 void	nfsd_cache_update(struct svc_rqst *, int, __be32 *);
+#ifdef CONFIG_NFSD_V4_1
+void	nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp);
+#else /* CONFIG_NFSD_V4_1 */
+static inline void nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp) {}
+#endif /* CONFIG_NFSD_V4_1 */
 
 #endif /* NFSCACHE_H */
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index feab6ec..8ca6a82 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -99,10 +99,23 @@ struct nfs4_callback {
 	struct rpc_clnt *       cb_client;
 };
 
+/* Maximum number of pages per slot cache entry */
+#define NFSD_PAGES_PER_SLOT	1
+
+struct nfsd4_cache_entry {
+	__be32		ce_status;
+	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
+	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
+	short		ce_resused;
+	int		ce_opcnt;
+	int		ce_rpchdrlen;
+};
+
 struct nfsd4_slot {
 	bool				sl_inuse;
 	struct nfsd4_session		*sl_session;
 	u32				sl_seqid;
+	struct nfsd4_cache_entry	sl_cache_entry;
 };
 
 struct nfsd4_session {
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 9e4d8db..cde8947 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -50,6 +50,8 @@ struct nfsd4_compound_state {
 	struct nfs4_stateowner	*replay_owner;
 	/* For sessions DRC */
 	struct nfsd4_slot	*slot;
+	__be32			*statp;
+	u32			status;
 };
 
 struct nfsd4_change_info {
@@ -490,6 +492,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 		struct nfsd4_setclientid_confirm *setclientid_confirm);
 #if defined(CONFIG_NFSD_V4_1)
+extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
+extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
 extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 struct nfsd4_exchange_id *);
-- 
1.6.2.1

Re: [PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions

[J. Bruce Fields]

This one scares me.

On Sat, Mar 28, 2009 at 11:32:32AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Cache all the result pages, including the rpc header in rq_respages[0],
> for a request in the slot table cache entry.
> 
> Cache the statp pointer from nfsd_dispatch which points into rq_respages[0]
> just past the rpc header. When setting a cache entry, calculate and save the
> length of the nfs data minus the rpc header for rq_respages[0].
> 
> When replaying a cache entry, replace the cached rpc header with the
> replayed request rpc result header, unless there is not enough room in the
> cached results first page. In that case, use the cached rpc header.
> 
> The sessions fore channel maxresponse size cached is set to NFSD_PAGES_PER_SLOT
> * PAGE_SIZE. For compounds we are cacheing with operations such as READDIR
> that use the xdr_buf->pages to hold data, we choose to cache the extra page of
> data rather than copying data from xdr_buf->pages into the xdr_buf->head page.
> 
> [nfsd41: limit cache to maxresponsesize_cached]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++++++++++++
>  fs/nfsd/nfssvc.c           |    4 +
>  include/linux/nfsd/cache.h |    5 ++
>  include/linux/nfsd/state.h |   13 ++++
>  include/linux/nfsd/xdr4.h  |    4 +
>  5 files changed, 168 insertions(+), 0 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 10eb67b..f0ce639 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -860,6 +860,148 @@ out_err:
>  }
>  
>  #if defined(CONFIG_NFSD_V4_1)
> +void
> +nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
> +{
> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
> +
> +	resp->cstate.statp = statp;
> +}
> +
> +/*
> + * Dereference the result pages.
> + */
> +static void
> +nfsd4_release_respages(struct page **respages, short resused)
> +{
> +	int page_no;
> +
> +	dprintk("--> %s\n", __func__);
> +	for (page_no = 0; page_no < resused; page_no++) {
> +		if (!respages[page_no])
> +			continue;
> +		put_page(respages[page_no]);
> +		respages[page_no] = NULL;
> +	}
> +}
> +
> +static void
> +nfsd4_move_pages(struct page **topages, struct page **frompages, short count)

s/move/copy/; we're not removing anything from the source.

> +{
> +	int page_no;

As a general matter of style, I'd rather any loop variable in a function
this short and simple be named "i".  "j" if you need another....

> +
> +	for (page_no = 0; page_no < count; page_no++) {
> +		topages[page_no] = frompages[page_no];
> +		if (!topages[page_no])
> +			continue;
> +		get_page(topages[page_no]);
> +	}
> +}
> +
> +/*
> + * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing the previous
> + * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where the total
> + * length of the XDR response is less than se_fmaxresp_cached
> + * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used for a
> + * of the reply (e.g. readdir).

That comment isn't very clear.

Is one page really sufficient?  Consider, for example, a 2-byte read
which spans a page boundary:

	first page: rpc header, compound header, putfh reply, etc.
	second page: 1st byte of read data
	third page: 2nd byte of read data
	fourth page: 2 bytes of padding, rest of reply.

That's for a reply of total length less than a page.

> + *
> + * Store the base and length of the rq_req.head[0] page
> + * of the NFSv4.1 data, just past the rpc header.
> + */
> +void
> +nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)

I find "set" a little vague.  How about "store"?

> +{
> +	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
> +	struct svc_rqst *rqstp = resp->rqstp;
> +	struct kvec *resv = &rqstp->rq_res.head[0];
> +
> +	dprintk("--> %s entry %p\n", __func__, entry);
> +
> +	/* Don't cache a failed OP_SEQUENCE */
> +	if (resp->opcnt == 1 && resp->cstate.status)
> +		return;
> +	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
> +	entry->ce_resused = rqstp->rq_resused;
> +	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
> +		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
> +	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
> +			 entry->ce_resused);
> +	entry->ce_status = resp->cstate.status;

Don't we need to track rq_res.page_base, page_len, etc.?  Try testing
replays of small unaligned reads.

> +	entry->ce_datav.iov_base = resp->cstate.statp;
> +	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
> +				(char *)page_address(rqstp->rq_respages[0]));
> +	entry->ce_opcnt = resp->opcnt;

Why do we need to save and restore the number of operations?

In general--I'd rather functions and data structures got introduced in
the same patch as their users; they're harder to judge on their own.

> +	/* Current request rpc header length*/
> +	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
> +				(char *)page_address(rqstp->rq_respages[0]);

I don't believe we need to save ce_rpchdrlen.

> +}
> +
> +/*
> + * Copy the cached NFSv4.1 reply skipping the cached rpc header into the
> + * replay result res.head[0] past the rpc header to end up with replay
> + * rpc header and cached NFSv4.1 reply.

This comment could be clearer; how about just:

	We keep the rpc header, but take the nfs reply from the reply
	cache.

?

> + */
> +static int
> +nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
> +			struct nfsd4_cache_entry *entry)
> +{
> +	struct svc_rqst *rqstp = resp->rqstp;
> +	struct kvec *resv = &resp->rqstp->rq_res.head[0];
> +	int len;
> +
> +	/* Current request rpc header length*/
> +	len = (char *)resp->cstate.statp -
> +			(char *)page_address(rqstp->rq_respages[0]);

Could write just resv->iov_base for for the second term there, I
beleive.

> +	if (entry->ce_datav.iov_len + len > PAGE_SIZE) {

This should depend on NFSD_MAX_PAGES_PER_SLOT, or something--we
shouldn't be hard-wiring the assumption that the maximum cached reply
size is PAGE_SIZE.

> +		dprintk("%s v41 cached reply too large (%Zd).\n", __func__,
> +			entry->ce_datav.iov_len);
> +		return 0;
> +	}
> +	/* copy the cached reply nfsd data past the current rpc header */
> +	memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_base,

That first argument could just be resp->cstate.statp.

> +		entry->ce_datav.iov_len);
> +	resv->iov_len = len + entry->ce_datav.iov_len;
> +	return 1;
> +}
> +
> +/*
> + * Keep the first page of the replay. Copy the NFSv4.1 data from the first
> + * cached page.  Replace any futher replay pages from the cache.
> + */
> +__be32
> +nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
> +{
> +	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
> +	__be32 status;
> +
> +	dprintk("--> %s entry %p\n", __func__, entry);
> +
> +
> +	if (!nfsd41_copy_replay_data(resp, entry)) {
> +		/*
> +		 * Not enough room to use the replay rpc header, send the
> +		 * cached header. Release all the allocated result pages.
> +		 */

No, we can't do this.  The protocol requires that we use the rpc header
from the replay.

--b.

> +		svc_free_res_pages(resp->rqstp);
> +		nfsd4_move_pages(resp->rqstp->rq_respages, entry->ce_respages,
> +			entry->ce_resused);
> +	} else {
> +		/* Release all but the first allocated result page */
> +
> +		resp->rqstp->rq_resused--;
> +		svc_free_res_pages(resp->rqstp);
> +
> +		nfsd4_move_pages(&resp->rqstp->rq_respages[1],
> +				 &entry->ce_respages[1],
> +				 entry->ce_resused - 1);
> +	}
> +
> +	resp->rqstp->rq_resused = entry->ce_resused;
> +	status = entry->ce_status;
> +
> +	return status;
> +}
> +
>  /*
>   * Set the exchange_id flags returned by the server.
>   */
> diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
> index ef0a368..b5168d1 100644
> --- a/fs/nfsd/nfssvc.c
> +++ b/fs/nfsd/nfssvc.c
> @@ -515,6 +515,10 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 *statp)
>  		+ rqstp->rq_res.head[0].iov_len;
>  	rqstp->rq_res.head[0].iov_len += sizeof(__be32);
>  
> +	/* NFSv4.1 DRC requires statp */
> +	if (rqstp->rq_vers == 4)
> +		nfsd4_set_statp(rqstp, statp);
> +
>  	/* Now call the procedure handler, and encode NFS status. */
>  	nfserr = proc->pc_func(rqstp, rqstp->rq_argp, rqstp->rq_resp);
>  	nfserr = map_new_errors(rqstp->rq_vers, nfserr);
> diff --git a/include/linux/nfsd/cache.h b/include/linux/nfsd/cache.h
> index 04b355c..57a83c7 100644
> --- a/include/linux/nfsd/cache.h
> +++ b/include/linux/nfsd/cache.h
> @@ -75,5 +75,10 @@ int	nfsd_reply_cache_init(void);
>  void	nfsd_reply_cache_shutdown(void);
>  int	nfsd_cache_lookup(struct svc_rqst *, int);
>  void	nfsd_cache_update(struct svc_rqst *, int, __be32 *);
> +#ifdef CONFIG_NFSD_V4_1
> +void	nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp);
> +#else /* CONFIG_NFSD_V4_1 */
> +static inline void nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp) {}
> +#endif /* CONFIG_NFSD_V4_1 */
>  
>  #endif /* NFSCACHE_H */
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index feab6ec..8ca6a82 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -99,10 +99,23 @@ struct nfs4_callback {
>  	struct rpc_clnt *       cb_client;
>  };
>  
> +/* Maximum number of pages per slot cache entry */
> +#define NFSD_PAGES_PER_SLOT	1
> +
> +struct nfsd4_cache_entry {
> +	__be32		ce_status;
> +	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
> +	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
> +	short		ce_resused;
> +	int		ce_opcnt;
> +	int		ce_rpchdrlen;
> +};
> +
>  struct nfsd4_slot {
>  	bool				sl_inuse;
>  	struct nfsd4_session		*sl_session;
>  	u32				sl_seqid;
> +	struct nfsd4_cache_entry	sl_cache_entry;
>  };
>  
>  struct nfsd4_session {
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 9e4d8db..cde8947 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -50,6 +50,8 @@ struct nfsd4_compound_state {
>  	struct nfs4_stateowner	*replay_owner;
>  	/* For sessions DRC */
>  	struct nfsd4_slot	*slot;
> +	__be32			*statp;
> +	u32			status;
>  };
>  
>  struct nfsd4_change_info {
> @@ -490,6 +492,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>  		struct nfsd4_compound_state *,
>  		struct nfsd4_setclientid_confirm *setclientid_confirm);
>  #if defined(CONFIG_NFSD_V4_1)
> +extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
> +extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
>  extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
>  		struct nfsd4_compound_state *,
>  struct nfsd4_exchange_id *);
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions

[Andy Adamson]

On Mar 31, 2009, at 7:03 PM, J. Bruce Fields wrote:

> This one scares me.
>
> On Sat, Mar 28, 2009 at 11:32:32AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Cache all the result pages, including the rpc header in  
>> rq_respages[0],
>> for a request in the slot table cache entry.
>>
>> Cache the statp pointer from nfsd_dispatch which points into  
>> rq_respages[0]
>> just past the rpc header. When setting a cache entry, calculate and  
>> save the
>> length of the nfs data minus the rpc header for rq_respages[0].
>>
>> When replaying a cache entry, replace the cached rpc header with the
>> replayed request rpc result header, unless there is not enough room  
>> in the
>> cached results first page. In that case, use the cached rpc header.
>>
>> The sessions fore channel maxresponse size cached is set to  
>> NFSD_PAGES_PER_SLOT
>> * PAGE_SIZE. For compounds we are cacheing with operations such as  
>> READDIR
>> that use the xdr_buf->pages to hold data, we choose to cache the  
>> extra page of
>> data rather than copying data from xdr_buf->pages into the xdr_buf- 
>> >head page.
>>
>> [nfsd41: limit cache to maxresponsesize_cached]
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>> fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++ 
>> ++++++++++
>> fs/nfsd/nfssvc.c           |    4 +
>> include/linux/nfsd/cache.h |    5 ++
>> include/linux/nfsd/state.h |   13 ++++
>> include/linux/nfsd/xdr4.h  |    4 +
>> 5 files changed, 168 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index 10eb67b..f0ce639 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -860,6 +860,148 @@ out_err:
>> }
>>
>> #if defined(CONFIG_NFSD_V4_1)
>> +void
>> +nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
>> +{
>> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>> +
>> +	resp->cstate.statp = statp;
>> +}
>> +
>> +/*
>> + * Dereference the result pages.
>> + */
>> +static void
>> +nfsd4_release_respages(struct page **respages, short resused)
>> +{
>> +	int page_no;
>> +
>> +	dprintk("--> %s\n", __func__);
>> +	for (page_no = 0; page_no < resused; page_no++) {
>> +		if (!respages[page_no])
>> +			continue;
>> +		put_page(respages[page_no]);
>> +		respages[page_no] = NULL;
>> +	}
>> +}
>> +
>> +static void
>> +nfsd4_move_pages(struct page **topages, struct page **frompages,  
>> short count)
>
> s/move/copy/; we're not removing anything from the source.
>
>> +{
>> +	int page_no;
>
> As a general matter of style, I'd rather any loop variable in a  
> function
> this short and simple be named "i".  "j" if you need another....
>
>> +
>> +	for (page_no = 0; page_no < count; page_no++) {
>> +		topages[page_no] = frompages[page_no];
>> +		if (!topages[page_no])
>> +			continue;
>> +		get_page(topages[page_no]);
>> +	}
>> +}
>> +
>> +/*
>> + * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing  
>> the previous
>> + * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where  
>> the total
>> + * length of the XDR response is less than se_fmaxresp_cached
>> + * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used  
>> for a
>> + * of the reply (e.g. readdir).
>
> That comment isn't very clear.
>
> Is one page really sufficient?  Consider, for example, a 2-byte read
> which spans a page boundary:
>
> 	first page: rpc header, compound header, putfh reply, etc.
> 	second page: 1st byte of read data
> 	third page: 2nd byte of read data
> 	fourth page: 2 bytes of padding, rest of reply.
>
> That's for a reply of total length less than a page.

I didn't realize the VFS returned read data in this manner. I thought  
a 2 byte read would end up as the first two bytes in the first page of  
the iovec presented to vfs_readv. Does the server actually send 4  
pages of data for a two byte read??

-->Andy
>
>
>> + *
>> + * Store the base and length of the rq_req.head[0] page
>> + * of the NFSv4.1 data, just past the rpc header.
>> + */
>> +void
>> +nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
>
> I find "set" a little vague.  How about "store"?
>
>> +{
>> +	struct nfsd4_cache_entry *entry = &resp->cstate.slot- 
>> >sl_cache_entry;
>> +	struct svc_rqst *rqstp = resp->rqstp;
>> +	struct kvec *resv = &rqstp->rq_res.head[0];
>> +
>> +	dprintk("--> %s entry %p\n", __func__, entry);
>> +
>> +	/* Don't cache a failed OP_SEQUENCE */
>> +	if (resp->opcnt == 1 && resp->cstate.status)
>> +		return;
>> +	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
>> +	entry->ce_resused = rqstp->rq_resused;
>> +	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
>> +		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
>> +	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
>> +			 entry->ce_resused);
>> +	entry->ce_status = resp->cstate.status;
>
> Don't we need to track rq_res.page_base, page_len, etc.?  Try testing
> replays of small unaligned reads.
>
>> +	entry->ce_datav.iov_base = resp->cstate.statp;
>> +	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp- 
>> >cstate.statp -
>> +				(char *)page_address(rqstp->rq_respages[0]));
>> +	entry->ce_opcnt = resp->opcnt;
>
> Why do we need to save and restore the number of operations?
>
> In general--I'd rather functions and data structures got introduced in
> the same patch as their users; they're harder to judge on their own.
>
>> +	/* Current request rpc header length*/
>> +	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
>> +				(char *)page_address(rqstp->rq_respages[0]);
>
> I don't believe we need to save ce_rpchdrlen.
>
>> +}
>> +
>> +/*
>> + * Copy the cached NFSv4.1 reply skipping the cached rpc header  
>> into the
>> + * replay result res.head[0] past the rpc header to end up with  
>> replay
>> + * rpc header and cached NFSv4.1 reply.
>
> This comment could be clearer; how about just:
>
> 	We keep the rpc header, but take the nfs reply from the reply
> 	cache.
>
> ?
>
>> + */
>> +static int
>> +nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
>> +			struct nfsd4_cache_entry *entry)
>> +{
>> +	struct svc_rqst *rqstp = resp->rqstp;
>> +	struct kvec *resv = &resp->rqstp->rq_res.head[0];
>> +	int len;
>> +
>> +	/* Current request rpc header length*/
>> +	len = (char *)resp->cstate.statp -
>> +			(char *)page_address(rqstp->rq_respages[0]);
>
> Could write just resv->iov_base for for the second term there, I
> beleive.
>
>> +	if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
>
> This should depend on NFSD_MAX_PAGES_PER_SLOT, or something--we
> shouldn't be hard-wiring the assumption that the maximum cached reply
> size is PAGE_SIZE.
>
>> +		dprintk("%s v41 cached reply too large (%Zd).\n", __func__,
>> +			entry->ce_datav.iov_len);
>> +		return 0;
>> +	}
>> +	/* copy the cached reply nfsd data past the current rpc header */
>> +	memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_base,
>
> That first argument could just be resp->cstate.statp.
>
>> +		entry->ce_datav.iov_len);
>> +	resv->iov_len = len + entry->ce_datav.iov_len;
>> +	return 1;
>> +}
>> +
>> +/*
>> + * Keep the first page of the replay. Copy the NFSv4.1 data from  
>> the first
>> + * cached page.  Replace any futher replay pages from the cache.
>> + */
>> +__be32
>> +nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
>> +{
>> +	struct nfsd4_cache_entry *entry = &resp->cstate.slot- 
>> >sl_cache_entry;
>> +	__be32 status;
>> +
>> +	dprintk("--> %s entry %p\n", __func__, entry);
>> +
>> +
>> +	if (!nfsd41_copy_replay_data(resp, entry)) {
>> +		/*
>> +		 * Not enough room to use the replay rpc header, send the
>> +		 * cached header. Release all the allocated result pages.
>> +		 */
>
> No, we can't do this.  The protocol requires that we use the rpc  
> header
> from the replay.
>
> --b.
>
>> +		svc_free_res_pages(resp->rqstp);
>> +		nfsd4_move_pages(resp->rqstp->rq_respages, entry->ce_respages,
>> +			entry->ce_resused);
>> +	} else {
>> +		/* Release all but the first allocated result page */
>> +
>> +		resp->rqstp->rq_resused--;
>> +		svc_free_res_pages(resp->rqstp);
>> +
>> +		nfsd4_move_pages(&resp->rqstp->rq_respages[1],
>> +				 &entry->ce_respages[1],
>> +				 entry->ce_resused - 1);
>> +	}
>> +
>> +	resp->rqstp->rq_resused = entry->ce_resused;
>> +	status = entry->ce_status;
>> +
>> +	return status;
>> +}
>> +
>> /*
>>  * Set the exchange_id flags returned by the server.
>>  */
>> diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
>> index ef0a368..b5168d1 100644
>> --- a/fs/nfsd/nfssvc.c
>> +++ b/fs/nfsd/nfssvc.c
>> @@ -515,6 +515,10 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32  
>> *statp)
>> 		+ rqstp->rq_res.head[0].iov_len;
>> 	rqstp->rq_res.head[0].iov_len += sizeof(__be32);
>>
>> +	/* NFSv4.1 DRC requires statp */
>> +	if (rqstp->rq_vers == 4)
>> +		nfsd4_set_statp(rqstp, statp);
>> +
>> 	/* Now call the procedure handler, and encode NFS status. */
>> 	nfserr = proc->pc_func(rqstp, rqstp->rq_argp, rqstp->rq_resp);
>> 	nfserr = map_new_errors(rqstp->rq_vers, nfserr);
>> diff --git a/include/linux/nfsd/cache.h b/include/linux/nfsd/cache.h
>> index 04b355c..57a83c7 100644
>> --- a/include/linux/nfsd/cache.h
>> +++ b/include/linux/nfsd/cache.h
>> @@ -75,5 +75,10 @@ int	nfsd_reply_cache_init(void);
>> void	nfsd_reply_cache_shutdown(void);
>> int	nfsd_cache_lookup(struct svc_rqst *, int);
>> void	nfsd_cache_update(struct svc_rqst *, int, __be32 *);
>> +#ifdef CONFIG_NFSD_V4_1
>> +void	nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp);
>> +#else /* CONFIG_NFSD_V4_1 */
>> +static inline void nfsd4_set_statp(struct svc_rqst *rqstp, __be32  
>> *statp) {}
>> +#endif /* CONFIG_NFSD_V4_1 */
>>
>> #endif /* NFSCACHE_H */
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index feab6ec..8ca6a82 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -99,10 +99,23 @@ struct nfs4_callback {
>> 	struct rpc_clnt *       cb_client;
>> };
>>
>> +/* Maximum number of pages per slot cache entry */
>> +#define NFSD_PAGES_PER_SLOT	1
>> +
>> +struct nfsd4_cache_entry {
>> +	__be32		ce_status;
>> +	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
>> +	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
>> +	short		ce_resused;
>> +	int		ce_opcnt;
>> +	int		ce_rpchdrlen;
>> +};
>> +
>> struct nfsd4_slot {
>> 	bool				sl_inuse;
>> 	struct nfsd4_session		*sl_session;
>> 	u32				sl_seqid;
>> +	struct nfsd4_cache_entry	sl_cache_entry;
>> };
>>
>> struct nfsd4_session {
>> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
>> index 9e4d8db..cde8947 100644
>> --- a/include/linux/nfsd/xdr4.h
>> +++ b/include/linux/nfsd/xdr4.h
>> @@ -50,6 +50,8 @@ struct nfsd4_compound_state {
>> 	struct nfs4_stateowner	*replay_owner;
>> 	/* For sessions DRC */
>> 	struct nfsd4_slot	*slot;
>> +	__be32			*statp;
>> +	u32			status;
>> };
>>
>> struct nfsd4_change_info {
>> @@ -490,6 +492,8 @@ extern __be32 nfsd4_setclientid_confirm(struct  
>> svc_rqst *rqstp,
>> 		struct nfsd4_compound_state *,
>> 		struct nfsd4_setclientid_confirm *setclientid_confirm);
>> #if defined(CONFIG_NFSD_V4_1)
>> +extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
>> +extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres  
>> *resp);
>> extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
>> 		struct nfsd4_compound_state *,
>> struct nfsd4_exchange_id *);
>> -- 
>> 1.6.2.1
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions

[J. Bruce Fields]

On Wed, Apr 01, 2009 at 02:23:11PM -0400, Andy Adamson wrote:
>
> On Mar 31, 2009, at 7:03 PM, J. Bruce Fields wrote:
>
>> This one scares me.
>>
>> On Sat, Mar 28, 2009 at 11:32:32AM +0300, Benny Halevy wrote:
>>> From: Andy Adamson <andros@netapp.com>
>>>
>>> Cache all the result pages, including the rpc header in  
>>> rq_respages[0],
>>> for a request in the slot table cache entry.
>>>
>>> Cache the statp pointer from nfsd_dispatch which points into  
>>> rq_respages[0]
>>> just past the rpc header. When setting a cache entry, calculate and  
>>> save the
>>> length of the nfs data minus the rpc header for rq_respages[0].
>>>
>>> When replaying a cache entry, replace the cached rpc header with the
>>> replayed request rpc result header, unless there is not enough room  
>>> in the
>>> cached results first page. In that case, use the cached rpc header.
>>>
>>> The sessions fore channel maxresponse size cached is set to  
>>> NFSD_PAGES_PER_SLOT
>>> * PAGE_SIZE. For compounds we are cacheing with operations such as  
>>> READDIR
>>> that use the xdr_buf->pages to hold data, we choose to cache the  
>>> extra page of
>>> data rather than copying data from xdr_buf->pages into the xdr_buf- 
>>> >head page.
>>>
>>> [nfsd41: limit cache to maxresponsesize_cached]
>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>> [nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>> ---
>>> fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++ 
>>> ++++++++++
>>> fs/nfsd/nfssvc.c           |    4 +
>>> include/linux/nfsd/cache.h |    5 ++
>>> include/linux/nfsd/state.h |   13 ++++
>>> include/linux/nfsd/xdr4.h  |    4 +
>>> 5 files changed, 168 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>>> index 10eb67b..f0ce639 100644
>>> --- a/fs/nfsd/nfs4state.c
>>> +++ b/fs/nfsd/nfs4state.c
>>> @@ -860,6 +860,148 @@ out_err:
>>> }
>>>
>>> #if defined(CONFIG_NFSD_V4_1)
>>> +void
>>> +nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
>>> +{
>>> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>>> +
>>> +	resp->cstate.statp = statp;
>>> +}
>>> +
>>> +/*
>>> + * Dereference the result pages.
>>> + */
>>> +static void
>>> +nfsd4_release_respages(struct page **respages, short resused)
>>> +{
>>> +	int page_no;
>>> +
>>> +	dprintk("--> %s\n", __func__);
>>> +	for (page_no = 0; page_no < resused; page_no++) {
>>> +		if (!respages[page_no])
>>> +			continue;
>>> +		put_page(respages[page_no]);
>>> +		respages[page_no] = NULL;
>>> +	}
>>> +}
>>> +
>>> +static void
>>> +nfsd4_move_pages(struct page **topages, struct page **frompages,  
>>> short count)
>>
>> s/move/copy/; we're not removing anything from the source.
>>
>>> +{
>>> +	int page_no;
>>
>> As a general matter of style, I'd rather any loop variable in a  
>> function
>> this short and simple be named "i".  "j" if you need another....
>>
>>> +
>>> +	for (page_no = 0; page_no < count; page_no++) {
>>> +		topages[page_no] = frompages[page_no];
>>> +		if (!topages[page_no])
>>> +			continue;
>>> +		get_page(topages[page_no]);
>>> +	}
>>> +}
>>> +
>>> +/*
>>> + * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing  
>>> the previous
>>> + * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where  
>>> the total
>>> + * length of the XDR response is less than se_fmaxresp_cached
>>> + * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used  
>>> for a
>>> + * of the reply (e.g. readdir).
>>
>> That comment isn't very clear.
>>
>> Is one page really sufficient?  Consider, for example, a 2-byte read
>> which spans a page boundary:
>>
>> 	first page: rpc header, compound header, putfh reply, etc.
>> 	second page: 1st byte of read data
>> 	third page: 2nd byte of read data
>> 	fourth page: 2 bytes of padding, rest of reply.
>>
>> That's for a reply of total length less than a page.
>
> I didn't realize the VFS returned read data in this manner. I thought a 2 
> byte read would end up as the first two bytes in the first page of the 
> iovec presented to vfs_readv. Does the server actually send 4 pages of 
> data for a two byte read??

Whoops, sorry, I forgot this question--email's coming out of my ears.

In nfsd_vfs_read() there are two cases, one using splice, and one readv.
Trace through it and see that in the splice case nfsd_splice_actor() is
called, which just takes references to pages in the page cache and sets
page_base and page_len appropriately.  The kvec passed into
nfsd4_encode_read isn't used in that case.

So, yes, if you do a 2-byte read from offset 1023 into a file (for
example), I believe you get a case like the above.

And the splice case is important to performance--we shouldn't just
bypass it for 4.1.

--b.

Re: [pnfs] [PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions

[William A. (Andy) Adamson]

On Thu, Apr 2, 2009 at 5:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> On Wed, Apr 01, 2009 at 02:23:11PM -0400, Andy Adamson wrote:
>>
>> On Mar 31, 2009, at 7:03 PM, J. Bruce Fields wrote:
>>
>>> This one scares me.
>>>
>>> On Sat, Mar 28, 2009 at 11:32:32AM +0300, Benny Halevy wrote:
>>>> From: Andy Adamson <andros@netapp.com>
>>>>
>>>> Cache all the result pages, including the rpc header in
>>>> rq_respages[0],
>>>> for a request in the slot table cache entry.
>>>>
>>>> Cache the statp pointer from nfsd_dispatch which points into
>>>> rq_respages[0]
>>>> just past the rpc header. When setting a cache entry, calculate and
>>>> save the
>>>> length of the nfs data minus the rpc header for rq_respages[0].
>>>>
>>>> When replaying a cache entry, replace the cached rpc header with the
>>>> replayed request rpc result header, unless there is not enough room
>>>> in the
>>>> cached results first page. In that case, use the cached rpc header.
>>>>
>>>> The sessions fore channel maxresponse size cached is set to
>>>> NFSD_PAGES_PER_SLOT
>>>> * PAGE_SIZE. For compounds we are cacheing with operations such as
>>>> READDIR
>>>> that use the xdr_buf->pages to hold data, we choose to cache the
>>>> extra page of
>>>> data rather than copying data from xdr_buf->pages into the xdr_buf-
>>>> >head page.
>>>>
>>>> [nfsd41: limit cache to maxresponsesize_cached]
>>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>>> [nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
>>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>>> ---
>>>> fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++
>>>> ++++++++++
>>>> fs/nfsd/nfssvc.c           |    4 +
>>>> include/linux/nfsd/cache.h |    5 ++
>>>> include/linux/nfsd/state.h |   13 ++++
>>>> include/linux/nfsd/xdr4.h  |    4 +
>>>> 5 files changed, 168 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>>>> index 10eb67b..f0ce639 100644
>>>> --- a/fs/nfsd/nfs4state.c
>>>> +++ b/fs/nfsd/nfs4state.c
>>>> @@ -860,6 +860,148 @@ out_err:
>>>> }
>>>>
>>>> #if defined(CONFIG_NFSD_V4_1)
>>>> +void
>>>> +nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
>>>> +{
>>>> +   struct nfsd4_compoundres *resp = rqstp->rq_resp;
>>>> +
>>>> +   resp->cstate.statp = statp;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Dereference the result pages.
>>>> + */
>>>> +static void
>>>> +nfsd4_release_respages(struct page **respages, short resused)
>>>> +{
>>>> +   int page_no;
>>>> +
>>>> +   dprintk("--> %s\n", __func__);
>>>> +   for (page_no = 0; page_no < resused; page_no++) {
>>>> +           if (!respages[page_no])
>>>> +                   continue;
>>>> +           put_page(respages[page_no]);
>>>> +           respages[page_no] = NULL;
>>>> +   }
>>>> +}
>>>> +
>>>> +static void
>>>> +nfsd4_move_pages(struct page **topages, struct page **frompages,
>>>> short count)
>>>
>>> s/move/copy/; we're not removing anything from the source.
>>>
>>>> +{
>>>> +   int page_no;
>>>
>>> As a general matter of style, I'd rather any loop variable in a
>>> function
>>> this short and simple be named "i".  "j" if you need another....
>>>
>>>> +
>>>> +   for (page_no = 0; page_no < count; page_no++) {
>>>> +           topages[page_no] = frompages[page_no];
>>>> +           if (!topages[page_no])
>>>> +                   continue;
>>>> +           get_page(topages[page_no]);
>>>> +   }
>>>> +}
>>>> +
>>>> +/*
>>>> + * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing
>>>> the previous
>>>> + * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where
>>>> the total
>>>> + * length of the XDR response is less than se_fmaxresp_cached
>>>> + * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used
>>>> for a
>>>> + * of the reply (e.g. readdir).
>>>
>>> That comment isn't very clear.
>>>
>>> Is one page really sufficient?  Consider, for example, a 2-byte read
>>> which spans a page boundary:
>>>
>>>      first page: rpc header, compound header, putfh reply, etc.
>>>      second page: 1st byte of read data
>>>      third page: 2nd byte of read data
>>>      fourth page: 2 bytes of padding, rest of reply.
>>>
>>> That's for a reply of total length less than a page.
>>
>> I didn't realize the VFS returned read data in this manner. I thought a 2
>> byte read would end up as the first two bytes in the first page of the
>> iovec presented to vfs_readv. Does the server actually send 4 pages of
>> data for a two byte read??
>
> Whoops, sorry, I forgot this question--email's coming out of my ears.

That's ok!

>
> In nfsd_vfs_read() there are two cases, one using splice, and one readv.
> Trace through it and see that in the splice case nfsd_splice_actor() is
> called, which just takes references to pages in the page cache and sets
> page_base and page_len appropriately.  The kvec passed into
> nfsd4_encode_read isn't used in that case.
>
> So, yes, if you do a 2-byte read from offset 1023 into a file (for
> example), I believe you get a case like the above.
>
> And the splice case is important to performance--we shouldn't just
> bypass it for 4.1.

OK. I've come to see that I've done this bas-akwards. I'm caching the
original page with rpc header and data, then copying the new rpc
header (which may not fit AKK) into the replay page  replacing the old
rpc header.

What I should be doing is saving the encoded operations (not including
the sequence op as you suggested) and copying the cached encoded
operations into the new reply page. This will also let us get past the
nfsd_splice_actor() case 'cause we can just cache the actual data (the
two bytes in the above example).

I probably won't be able to reliably code all this tonight, although I
will give it a whirl !!

I hope that if I address the other comments and keep the current
bas-akwards code you will consider accepting it to allow me to bug fix
it in the 2.6.30-rcX series. I promise to do so.

-->Andy
>
> --b.
> _______________________________________________
> pNFS mailing list
> pNFS@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
>

Re: [pnfs] [PATCH v2 19/47] nfsd41: DRC save, restore, and clear functions

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 05:29:10PM -0400, William A. (Andy) Adamson wrote:
> On Thu, Apr 2, 2009 at 5:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> > On Wed, Apr 01, 2009 at 02:23:11PM -0400, Andy Adamson wrote:
> >>
> >> On Mar 31, 2009, at 7:03 PM, J. Bruce Fields wrote:
> >>
> >>> This one scares me.
> >>>
> >>> On Sat, Mar 28, 2009 at 11:32:32AM +0300, Benny Halevy wrote:
> >>>> From: Andy Adamson <andros@netapp.com>
> >>>>
> >>>> Cache all the result pages, including the rpc header in
> >>>> rq_respages[0],
> >>>> for a request in the slot table cache entry.
> >>>>
> >>>> Cache the statp pointer from nfsd_dispatch which points into
> >>>> rq_respages[0]
> >>>> just past the rpc header. When setting a cache entry, calculate and
> >>>> save the
> >>>> length of the nfs data minus the rpc header for rq_respages[0].
> >>>>
> >>>> When replaying a cache entry, replace the cached rpc header with the
> >>>> replayed request rpc result header, unless there is not enough room
> >>>> in the
> >>>> cached results first page. In that case, use the cached rpc header.
> >>>>
> >>>> The sessions fore channel maxresponse size cached is set to
> >>>> NFSD_PAGES_PER_SLOT
> >>>> * PAGE_SIZE. For compounds we are cacheing with operations such as
> >>>> READDIR
> >>>> that use the xdr_buf->pages to hold data, we choose to cache the
> >>>> extra page of
> >>>> data rather than copying data from xdr_buf->pages into the xdr_buf-
> >>>> >head page.
> >>>>
> >>>> [nfsd41: limit cache to maxresponsesize_cached]
> >>>> Signed-off-by: Andy Adamson <andros@netapp.com>
> >>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >>>> [nfsd41: mv nfsd4_set_statp under CONFIG_NFSD_V4_1]
> >>>> Signed-off-by: Andy Adamson <andros@netapp.com>
> >>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >>>> ---
> >>>> fs/nfsd/nfs4state.c        |  142 ++++++++++++++++++++++++++++++++++
> >>>> ++++++++++
> >>>> fs/nfsd/nfssvc.c           |    4 +
> >>>> include/linux/nfsd/cache.h |    5 ++
> >>>> include/linux/nfsd/state.h |   13 ++++
> >>>> include/linux/nfsd/xdr4.h  |    4 +
> >>>> 5 files changed, 168 insertions(+), 0 deletions(-)
> >>>>
> >>>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> >>>> index 10eb67b..f0ce639 100644
> >>>> --- a/fs/nfsd/nfs4state.c
> >>>> +++ b/fs/nfsd/nfs4state.c
> >>>> @@ -860,6 +860,148 @@ out_err:
> >>>> }
> >>>>
> >>>> #if defined(CONFIG_NFSD_V4_1)
> >>>> +void
> >>>> +nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
> >>>> +{
> >>>> +   struct nfsd4_compoundres *resp = rqstp->rq_resp;
> >>>> +
> >>>> +   resp->cstate.statp = statp;
> >>>> +}
> >>>> +
> >>>> +/*
> >>>> + * Dereference the result pages.
> >>>> + */
> >>>> +static void
> >>>> +nfsd4_release_respages(struct page **respages, short resused)
> >>>> +{
> >>>> +   int page_no;
> >>>> +
> >>>> +   dprintk("--> %s\n", __func__);
> >>>> +   for (page_no = 0; page_no < resused; page_no++) {
> >>>> +           if (!respages[page_no])
> >>>> +                   continue;
> >>>> +           put_page(respages[page_no]);
> >>>> +           respages[page_no] = NULL;
> >>>> +   }
> >>>> +}
> >>>> +
> >>>> +static void
> >>>> +nfsd4_move_pages(struct page **topages, struct page **frompages,
> >>>> short count)
> >>>
> >>> s/move/copy/; we're not removing anything from the source.
> >>>
> >>>> +{
> >>>> +   int page_no;
> >>>
> >>> As a general matter of style, I'd rather any loop variable in a
> >>> function
> >>> this short and simple be named "i".  "j" if you need another....
> >>>
> >>>> +
> >>>> +   for (page_no = 0; page_no < count; page_no++) {
> >>>> +           topages[page_no] = frompages[page_no];
> >>>> +           if (!topages[page_no])
> >>>> +                   continue;
> >>>> +           get_page(topages[page_no]);
> >>>> +   }
> >>>> +}
> >>>> +
> >>>> +/*
> >>>> + * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing
> >>>> the previous
> >>>> + * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where
> >>>> the total
> >>>> + * length of the XDR response is less than se_fmaxresp_cached
> >>>> + * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used
> >>>> for a
> >>>> + * of the reply (e.g. readdir).
> >>>
> >>> That comment isn't very clear.
> >>>
> >>> Is one page really sufficient?  Consider, for example, a 2-byte read
> >>> which spans a page boundary:
> >>>
> >>>      first page: rpc header, compound header, putfh reply, etc.
> >>>      second page: 1st byte of read data
> >>>      third page: 2nd byte of read data
> >>>      fourth page: 2 bytes of padding, rest of reply.
> >>>
> >>> That's for a reply of total length less than a page.
> >>
> >> I didn't realize the VFS returned read data in this manner. I thought a 2
> >> byte read would end up as the first two bytes in the first page of the
> >> iovec presented to vfs_readv. Does the server actually send 4 pages of
> >> data for a two byte read??
> >
> > Whoops, sorry, I forgot this question--email's coming out of my ears.
> 
> That's ok!
> 
> >
> > In nfsd_vfs_read() there are two cases, one using splice, and one readv.
> > Trace through it and see that in the splice case nfsd_splice_actor() is
> > called, which just takes references to pages in the page cache and sets
> > page_base and page_len appropriately.  The kvec passed into
> > nfsd4_encode_read isn't used in that case.
> >
> > So, yes, if you do a 2-byte read from offset 1023 into a file (for
> > example), I believe you get a case like the above.
> >
> > And the splice case is important to performance--we shouldn't just
> > bypass it for 4.1.
> 
> OK. I've come to see that I've done this bas-akwards. I'm caching the
> original page with rpc header and data, then copying the new rpc
> header (which may not fit AKK) into the replay page  replacing the old
> rpc header.
> 
> What I should be doing is saving the encoded operations (not including
> the sequence op as you suggested) and copying the cached encoded
> operations into the new reply page. This will also let us get past the
> nfsd_splice_actor() case 'cause we can just cache the actual data (the
> two bytes in the above example).

I think that's right....

> I probably won't be able to reliably code all this tonight, although I
> will give it a whirl !!
> 
> I hope that if I address the other comments and keep the current
> bas-akwards code you will consider accepting it to allow me to bug fix
> it in the 2.6.30-rcX series. I promise to do so.

I won't worry--it would be better to stick to trivial fixes and just
send me what you guys have now.

--b.

[PATCH v2 20/47] nfsd41: hard page limit for DRC

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Use no more than 1/128th of the number of free pages at nfsd startup for the
v4.1 DRC.

This is an arbitrary default which should probably end up under the control
of an administrator.

Signed-off-by: Andy Adamson <andros@netapp.com>
[moved added fields in struct svc_serv under CONFIG_NFSD_V4_1]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[fix set_max_drc calculation of sv_drc_max_pages]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfssvc.c           |   25 +++++++++++++++++++++++++
 include/linux/nfsd/nfsd.h  |    7 +++++++
 include/linux/sunrpc/svc.h |    4 ++++
 3 files changed, 36 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index b5168d1..ab60320 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -22,6 +22,7 @@
 #include <linux/freezer.h>
 #include <linux/fs_struct.h>
 #include <linux/kthread.h>
+#include <linux/swap.h>
 
 #include <linux/sunrpc/types.h>
 #include <linux/sunrpc/stats.h>
@@ -197,6 +198,28 @@ void nfsd_reset_versions(void)
 	}
 }
 
+/*
+ * Each session guarantees a negotiated per slot memory cache for replies
+ * which in turn consumes memory beyond the v2/v3/v4.0 server. A dedicated
+ * NFSv4.1 server might want to use more memory for a DRC than a machine
+ * with mutiple services.
+ *
+ * Impose a hard limit on the number of pages for the DRC which varies
+ * according to the machines free pages. This is of course only a default.
+ *
+ * For now this is a #defined shift which could be under admin control
+ * in the future.
+ */
+static void set_max_drc(void)
+{
+#ifdef CONFIG_NFSD_V4_1
+	nfsd_serv->sv_drc_max_pages = nr_free_buffer_pages()
+						>> NFSD_DRC_SIZE_SHIFT;
+	nfsd_serv->sv_drc_pages_used = 0;
+	dprintk("%s svc_drc_max_pages %u\n", __func__,
+		nfsd_serv->sv_drc_max_pages);
+#endif /* CONFIG_NFSD_V4_1 */
+}
 
 int nfsd_create_serv(void)
 {
@@ -229,6 +252,8 @@ int nfsd_create_serv(void)
 				      nfsd_last_thread, nfsd, THIS_MODULE);
 	if (nfsd_serv == NULL)
 		err = -ENOMEM;
+	else
+		set_max_drc();
 
 	do_gettimeofday(&nfssvc_boot);		/* record boot time */
 	return err;
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index ab9616d..3bccec3 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -361,6 +361,13 @@ extern struct timeval	nfssvc_boot;
  | FATTR4_WORD1_TIME_DELTA   | FATTR4_WORD1_TIME_METADATA    \
  | FATTR4_WORD1_TIME_MODIFY     | FATTR4_WORD1_TIME_MODIFY_SET | FATTR4_WORD1_MOUNTED_ON_FILEID)
 
+#if defined(CONFIG_NFSD_V4_1)
+
+/* The percent of nr_free_buffer_pages used by the V4.1 server DRC */
+#define NFSD_DRC_SIZE_SHIFT	7
+
+#endif /* CONFIG_NFSD_V4_1 */
+
 /* These will return ERR_INVAL if specified in GETATTR or READDIR. */
 #define NFSD_WRITEONLY_ATTRS_WORD1							    \
 (FATTR4_WORD1_TIME_ACCESS_SET   | FATTR4_WORD1_TIME_MODIFY_SET)
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 815dd58..c4f0e1e 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -95,6 +95,10 @@ struct svc_serv {
 	struct module *		sv_module;	/* optional module to count when
 						 * adding threads */
 	svc_thread_fn		sv_function;	/* main function for threads */
+#if defined(CONFIG_NFSD_V4_1)
+	unsigned int		sv_drc_max_pages; /* Total pages for DRC */
+	unsigned int		sv_drc_pages_used;/* DRC pages used */
+#endif /* CONFIG_NFSD_V4_1 */
 };
 
 /*
-- 
1.6.2.1

[PATCH v2 21/47] nfsd41: nfsd DRC logic

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Replay a request in nfsd4_sequence.
Add a minorversion to struct nfsd4_compound_state.

Pass the current slot to nfs4svc_encode_compound res via struct
nfsd4_compoundres to set an NFSv4.1 DRC entry.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |    7 +++++++
 fs/nfsd/nfs4state.c       |    6 ++++++
 fs/nfsd/nfs4xdr.c         |   13 +++++++++++++
 include/linux/nfsd/xdr4.h |    1 +
 4 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index e703ac2..bdbeb87 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -920,6 +920,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 			BUG_ON(op->status == nfs_ok);
 
 encode_op:
+		/* Only from SEQUENCE or CREATE_SESSION */
+		if (resp->cstate.status == nfserr_replay_cache) {
+			dprintk("%s NFS4.1 replay from cache\n", __func__);
+			status = op->status;
+			goto out;
+		}
 		if (op->status == nfserr_replay_me) {
 			op->replay = &cstate->replay_owner->so_replay;
 			nfsd4_encode_replay(resp, op);
@@ -948,6 +954,7 @@ encode_op:
 		status = nfserr_jukebox;
 	}
 
+	resp->cstate.status = status;
 	fh_put(&resp->cstate.current_fh);
 	fh_put(&resp->cstate.save_fh);
 	BUG_ON(resp->cstate.replay_owner);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f0ce639..07c869d 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -997,6 +997,8 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
 	}
 
 	resp->rqstp->rq_resused = entry->ce_resused;
+	resp->opcnt = entry->ce_opcnt;
+	resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
 	status = entry->ce_status;
 
 	return status;
@@ -1217,6 +1219,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	status = check_slot_seqid(seq->seqid, slot);
 	if (status == nfserr_replay_cache) {
 		cstate->slot = slot;
+		/* Return the cached reply status and set cstate->status
+		 * for nfsd4_svc_encode_compoundres processing*/
+		status = nfsd4_replay_cache_entry(resp);
+		cstate->status = nfserr_replay_cache;
 		goto replay_cache;
 	}
 	if (status)
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index c6b490e..57afb33 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3059,6 +3059,19 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 		iov = &rqstp->rq_res.head[0];
 	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
 	BUG_ON(iov->iov_len > PAGE_SIZE);
+#ifdef CONFIG_NFSD_V4_1
+	if (resp->cstate.slot != NULL) {
+		if (resp->cstate.status == nfserr_replay_cache) {
+			iov->iov_len = resp->cstate.iovlen;
+		} else {
+			nfsd4_set_cache_entry(resp);
+			dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
+			resp->cstate.slot->sl_inuse = 0;
+		}
+		if (resp->cstate.slot->sl_session)
+			nfsd4_put_session(resp->cstate.slot->sl_session);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 	return 1;
 }
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index cde8947..5c0d376 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -51,6 +51,7 @@ struct nfsd4_compound_state {
 	/* For sessions DRC */
 	struct nfsd4_slot	*slot;
 	__be32			*statp;
+	size_t			iovlen;
 	u32			status;
 };
 
-- 
1.6.2.1

Re: [PATCH v2 21/47] nfsd41: nfsd DRC logic

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:44AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Replay a request in nfsd4_sequence.
> Add a minorversion to struct nfsd4_compound_state.
> 
> Pass the current slot to nfs4svc_encode_compound res via struct
> nfsd4_compoundres to set an NFSv4.1 DRC entry.
> 
> Signed-off-by: Andy Adamson<andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: use bool inuse for slot state]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4proc.c        |    7 +++++++
>  fs/nfsd/nfs4state.c       |    6 ++++++
>  fs/nfsd/nfs4xdr.c         |   13 +++++++++++++
>  include/linux/nfsd/xdr4.h |    1 +
>  4 files changed, 27 insertions(+), 0 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index e703ac2..bdbeb87 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -920,6 +920,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  			BUG_ON(op->status == nfs_ok);
>  
>  encode_op:
> +		/* Only from SEQUENCE or CREATE_SESSION */
> +		if (resp->cstate.status == nfserr_replay_cache) {
> +			dprintk("%s NFS4.1 replay from cache\n", __func__);
> +			status = op->status;
> +			goto out;
> +		}
>  		if (op->status == nfserr_replay_me) {
>  			op->replay = &cstate->replay_owner->so_replay;
>  			nfsd4_encode_replay(resp, op);
> @@ -948,6 +954,7 @@ encode_op:
>  		status = nfserr_jukebox;
>  	}
>  
> +	resp->cstate.status = status;
>  	fh_put(&resp->cstate.current_fh);
>  	fh_put(&resp->cstate.save_fh);
>  	BUG_ON(resp->cstate.replay_owner);
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index f0ce639..07c869d 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -997,6 +997,8 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
>  	}
>  
>  	resp->rqstp->rq_resused = entry->ce_resused;
> +	resp->opcnt = entry->ce_opcnt;
> +	resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
>  	status = entry->ce_status;
>  
>  	return status;
> @@ -1217,6 +1219,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>  	status = check_slot_seqid(seq->seqid, slot);
>  	if (status == nfserr_replay_cache) {
>  		cstate->slot = slot;
> +		/* Return the cached reply status and set cstate->status
> +		 * for nfsd4_svc_encode_compoundres processing*/
> +		status = nfsd4_replay_cache_entry(resp);
> +		cstate->status = nfserr_replay_cache;
>  		goto replay_cache;
>  	}
>  	if (status)
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index c6b490e..57afb33 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -3059,6 +3059,19 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
>  		iov = &rqstp->rq_res.head[0];
>  	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
>  	BUG_ON(iov->iov_len > PAGE_SIZE);
> +#ifdef CONFIG_NFSD_V4_1
> +	if (resp->cstate.slot != NULL) {
> +		if (resp->cstate.status == nfserr_replay_cache) {
> +			iov->iov_len = resp->cstate.iovlen;
> +		} else {
> +			nfsd4_set_cache_entry(resp);
> +			dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
> +			resp->cstate.slot->sl_inuse = 0;
> +		}
> +		if (resp->cstate.slot->sl_session)

There's no way that sl_session could ever be NULL, so this check is
unneeded.

While we're at it: let's just eliminate the sl_session pointer from the
slot.  I think all that would be needed would be a pointer to the
session from the cstate, in addition to the pointer to the slot.

--b.

> +			nfsd4_put_session(resp->cstate.slot->sl_session);
> +	}
> +#endif /* CONFIG_NFSD_V4_1 */
>  	return 1;
>  }
>  
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index cde8947..5c0d376 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -51,6 +51,7 @@ struct nfsd4_compound_state {
>  	/* For sessions DRC */
>  	struct nfsd4_slot	*slot;
>  	__be32			*statp;
> +	size_t			iovlen;
>  	u32			status;
>  };
>  
> -- 
> 1.6.2.1
> 

Re: [pnfs] [PATCH v2 21/47] nfsd41: nfsd DRC logic

[William A. (Andy) Adamson]

On Tue, Mar 31, 2009 at 3:30 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:32:44AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Replay a request in nfsd4_sequence.
>> Add a minorversion to struct nfsd4_compound_state.
>>
>> Pass the current slot to nfs4svc_encode_compound res via struct
>> nfsd4_compoundres to set an NFSv4.1 DRC entry.
>>
>> Signed-off-by: Andy Adamson<andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [nfsd41: use bool inuse for slot state]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4proc.c        |    7 +++++++
>>  fs/nfsd/nfs4state.c       |    6 ++++++
>>  fs/nfsd/nfs4xdr.c         |   13 +++++++++++++
>>  include/linux/nfsd/xdr4.h |    1 +
>>  4 files changed, 27 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>> index e703ac2..bdbeb87 100644
>> --- a/fs/nfsd/nfs4proc.c
>> +++ b/fs/nfsd/nfs4proc.c
>> @@ -920,6 +920,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>>                       BUG_ON(op->status == nfs_ok);
>>
>>  encode_op:
>> +             /* Only from SEQUENCE or CREATE_SESSION */
>> +             if (resp->cstate.status == nfserr_replay_cache) {
>> +                     dprintk("%s NFS4.1 replay from cache\n", __func__);
>> +                     status = op->status;
>> +                     goto out;
>> +             }
>>               if (op->status == nfserr_replay_me) {
>>                       op->replay = &cstate->replay_owner->so_replay;
>>                       nfsd4_encode_replay(resp, op);
>> @@ -948,6 +954,7 @@ encode_op:
>>               status = nfserr_jukebox;
>>       }
>>
>> +     resp->cstate.status = status;
>>       fh_put(&resp->cstate.current_fh);
>>       fh_put(&resp->cstate.save_fh);
>>       BUG_ON(resp->cstate.replay_owner);
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index f0ce639..07c869d 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -997,6 +997,8 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
>>       }
>>
>>       resp->rqstp->rq_resused = entry->ce_resused;
>> +     resp->opcnt = entry->ce_opcnt;
>> +     resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
>>       status = entry->ce_status;
>>
>>       return status;
>> @@ -1217,6 +1219,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>>       status = check_slot_seqid(seq->seqid, slot);
>>       if (status == nfserr_replay_cache) {
>>               cstate->slot = slot;
>> +             /* Return the cached reply status and set cstate->status
>> +              * for nfsd4_svc_encode_compoundres processing*/
>> +             status = nfsd4_replay_cache_entry(resp);
>> +             cstate->status = nfserr_replay_cache;
>>               goto replay_cache;
>>       }
>>       if (status)
>> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
>> index c6b490e..57afb33 100644
>> --- a/fs/nfsd/nfs4xdr.c
>> +++ b/fs/nfsd/nfs4xdr.c
>> @@ -3059,6 +3059,19 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
>>               iov = &rqstp->rq_res.head[0];
>>       iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
>>       BUG_ON(iov->iov_len > PAGE_SIZE);
>> +#ifdef CONFIG_NFSD_V4_1
>> +     if (resp->cstate.slot != NULL) {
>> +             if (resp->cstate.status == nfserr_replay_cache) {
>> +                     iov->iov_len = resp->cstate.iovlen;
>> +             } else {
>> +                     nfsd4_set_cache_entry(resp);
>> +                     dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
>> +                     resp->cstate.slot->sl_inuse = 0;
>> +             }
>> +             if (resp->cstate.slot->sl_session)
>
> There's no way that sl_session could ever be NULL, so this check is
> unneeded.

There should have been a comment.
CREATE_SESSION sets cstate->slot using the nfs4_client->cl_slot, a
single slot which it uses for the CREATE_SESSION replay cache. It does
not have a session.


>
> While we're at it: let's just eliminate the sl_session pointer from the
> slot.  I think all that would be needed would be a pointer to the
> session from the cstate, in addition to the pointer to the slot.

OK.

>
> --b.
>
>> +                     nfsd4_put_session(resp->cstate.slot->sl_session);
>> +     }
>> +#endif /* CONFIG_NFSD_V4_1 */
>>       return 1;
>>  }
>>
>> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
>> index cde8947..5c0d376 100644
>> --- a/include/linux/nfsd/xdr4.h
>> +++ b/include/linux/nfsd/xdr4.h
>> @@ -51,6 +51,7 @@ struct nfsd4_compound_state {
>>       /* For sessions DRC */
>>       struct nfsd4_slot       *slot;
>>       __be32                  *statp;
>> +     size_t                  iovlen;
>>       u32                     status;
>>  };
>>
>> --
>> 1.6.2.1
>>
> _______________________________________________
> pNFS mailing list
> pNFS@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
>

Re: [pnfs] [PATCH v2 21/47] nfsd41: nfsd DRC logic

[J. Bruce Fields]

On Wed, Apr 01, 2009 at 03:01:53PM -0400, William A. (Andy) Adamson wrote:
> On Tue, Mar 31, 2009 at 3:30 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:32:44AM +0300, Benny Halevy wrote:
> >> From: Andy Adamson <andros@netapp.com>
> >>
> >> Replay a request in nfsd4_sequence.
> >> Add a minorversion to struct nfsd4_compound_state.
> >>
> >> Pass the current slot to nfs4svc_encode_compound res via struct
> >> nfsd4_compoundres to set an NFSv4.1 DRC entry.
> >>
> >> Signed-off-by: Andy Adamson<andros@netapp.com>
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> [nfsd41: use bool inuse for slot state]
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> ---
> >>  fs/nfsd/nfs4proc.c        |    7 +++++++
> >>  fs/nfsd/nfs4state.c       |    6 ++++++
> >>  fs/nfsd/nfs4xdr.c         |   13 +++++++++++++
> >>  include/linux/nfsd/xdr4.h |    1 +
> >>  4 files changed, 27 insertions(+), 0 deletions(-)
> >>
> >> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> >> index e703ac2..bdbeb87 100644
> >> --- a/fs/nfsd/nfs4proc.c
> >> +++ b/fs/nfsd/nfs4proc.c
> >> @@ -920,6 +920,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
> >>                       BUG_ON(op->status == nfs_ok);
> >>
> >>  encode_op:
> >> +             /* Only from SEQUENCE or CREATE_SESSION */
> >> +             if (resp->cstate.status == nfserr_replay_cache) {
> >> +                     dprintk("%s NFS4.1 replay from cache\n", __func__);
> >> +                     status = op->status;
> >> +                     goto out;
> >> +             }
> >>               if (op->status == nfserr_replay_me) {
> >>                       op->replay = &cstate->replay_owner->so_replay;
> >>                       nfsd4_encode_replay(resp, op);
> >> @@ -948,6 +954,7 @@ encode_op:
> >>               status = nfserr_jukebox;
> >>       }
> >>
> >> +     resp->cstate.status = status;
> >>       fh_put(&resp->cstate.current_fh);
> >>       fh_put(&resp->cstate.save_fh);
> >>       BUG_ON(resp->cstate.replay_owner);
> >> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> >> index f0ce639..07c869d 100644
> >> --- a/fs/nfsd/nfs4state.c
> >> +++ b/fs/nfsd/nfs4state.c
> >> @@ -997,6 +997,8 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
> >>       }
> >>
> >>       resp->rqstp->rq_resused = entry->ce_resused;
> >> +     resp->opcnt = entry->ce_opcnt;
> >> +     resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
> >>       status = entry->ce_status;
> >>
> >>       return status;
> >> @@ -1217,6 +1219,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
> >>       status = check_slot_seqid(seq->seqid, slot);
> >>       if (status == nfserr_replay_cache) {
> >>               cstate->slot = slot;
> >> +             /* Return the cached reply status and set cstate->status
> >> +              * for nfsd4_svc_encode_compoundres processing*/
> >> +             status = nfsd4_replay_cache_entry(resp);
> >> +             cstate->status = nfserr_replay_cache;
> >>               goto replay_cache;
> >>       }
> >>       if (status)
> >> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> >> index c6b490e..57afb33 100644
> >> --- a/fs/nfsd/nfs4xdr.c
> >> +++ b/fs/nfsd/nfs4xdr.c
> >> @@ -3059,6 +3059,19 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
> >>               iov = &rqstp->rq_res.head[0];
> >>       iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
> >>       BUG_ON(iov->iov_len > PAGE_SIZE);
> >> +#ifdef CONFIG_NFSD_V4_1
> >> +     if (resp->cstate.slot != NULL) {
> >> +             if (resp->cstate.status == nfserr_replay_cache) {
> >> +                     iov->iov_len = resp->cstate.iovlen;
> >> +             } else {
> >> +                     nfsd4_set_cache_entry(resp);
> >> +                     dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
> >> +                     resp->cstate.slot->sl_inuse = 0;
> >> +             }
> >> +             if (resp->cstate.slot->sl_session)
> >
> > There's no way that sl_session could ever be NULL, so this check is
> > unneeded.
> 
> There should have been a comment.
> CREATE_SESSION sets cstate->slot using the nfs4_client->cl_slot, a
> single slot which it uses for the CREATE_SESSION replay cache. It does
> not have a session.

Hm.  In that case, there's no reference count on the client that holds
the cl_slot structure, so is there anything preventing the cl_slot from
being freed out from under us  by a concurrent expire_client()?

If that's correct: I'm not sure how to solve that problem with a minimum
of special-casing.

--b.

> 
> 
> >
> > While we're at it: let's just eliminate the sl_session pointer from the
> > slot.  I think all that would be needed would be a pointer to the
> > session from the cstate, in addition to the pointer to the slot.
> 
> OK.
> 
> >
> > --b.
> >
> >> +                     nfsd4_put_session(resp->cstate.slot->sl_session);
> >> +     }
> >> +#endif /* CONFIG_NFSD_V4_1 */
> >>       return 1;
> >>  }
> >>
> >> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> >> index cde8947..5c0d376 100644
> >> --- a/include/linux/nfsd/xdr4.h
> >> +++ b/include/linux/nfsd/xdr4.h
> >> @@ -51,6 +51,7 @@ struct nfsd4_compound_state {
> >>       /* For sessions DRC */
> >>       struct nfsd4_slot       *slot;
> >>       __be32                  *statp;
> >> +     size_t                  iovlen;
> >>       u32                     status;
> >>  };
> >>
> >> --
> >> 1.6.2.1
> >>
> > _______________________________________________
> > pNFS mailing list
> > pNFS@linux-nfs.org
> > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
> >

Re: [pnfs] [PATCH v2 21/47] nfsd41: nfsd DRC logic

[William A. (Andy) Adamson]

On Wed, Apr 1, 2009 at 3:10 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> On Wed, Apr 01, 2009 at 03:01:53PM -0400, William A. (Andy) Adamson wrote:
>> On Tue, Mar 31, 2009 at 3:30 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>> > On Sat, Mar 28, 2009 at 11:32:44AM +0300, Benny Halevy wrote:
>> >> From: Andy Adamson <andros@netapp.com>
>> >>
>> >> Replay a request in nfsd4_sequence.
>> >> Add a minorversion to struct nfsd4_compound_state.
>> >>
>> >> Pass the current slot to nfs4svc_encode_compound res via struct
>> >> nfsd4_compoundres to set an NFSv4.1 DRC entry.
>> >>
>> >> Signed-off-by: Andy Adamson<andros@netapp.com>
>> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> >> [nfsd41: use bool inuse for slot state]
>> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> >> ---
>> >>  fs/nfsd/nfs4proc.c        |    7 +++++++
>> >>  fs/nfsd/nfs4state.c       |    6 ++++++
>> >>  fs/nfsd/nfs4xdr.c         |   13 +++++++++++++
>> >>  include/linux/nfsd/xdr4.h |    1 +
>> >>  4 files changed, 27 insertions(+), 0 deletions(-)
>> >>
>> >> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>> >> index e703ac2..bdbeb87 100644
>> >> --- a/fs/nfsd/nfs4proc.c
>> >> +++ b/fs/nfsd/nfs4proc.c
>> >> @@ -920,6 +920,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>> >>                       BUG_ON(op->status == nfs_ok);
>> >>
>> >>  encode_op:
>> >> +             /* Only from SEQUENCE or CREATE_SESSION */
>> >> +             if (resp->cstate.status == nfserr_replay_cache) {
>> >> +                     dprintk("%s NFS4.1 replay from cache\n", __func__);
>> >> +                     status = op->status;
>> >> +                     goto out;
>> >> +             }
>> >>               if (op->status == nfserr_replay_me) {
>> >>                       op->replay = &cstate->replay_owner->so_replay;
>> >>                       nfsd4_encode_replay(resp, op);
>> >> @@ -948,6 +954,7 @@ encode_op:
>> >>               status = nfserr_jukebox;
>> >>       }
>> >>
>> >> +     resp->cstate.status = status;
>> >>       fh_put(&resp->cstate.current_fh);
>> >>       fh_put(&resp->cstate.save_fh);
>> >>       BUG_ON(resp->cstate.replay_owner);
>> >> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> >> index f0ce639..07c869d 100644
>> >> --- a/fs/nfsd/nfs4state.c
>> >> +++ b/fs/nfsd/nfs4state.c
>> >> @@ -997,6 +997,8 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
>> >>       }
>> >>
>> >>       resp->rqstp->rq_resused = entry->ce_resused;
>> >> +     resp->opcnt = entry->ce_opcnt;
>> >> +     resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
>> >>       status = entry->ce_status;
>> >>
>> >>       return status;
>> >> @@ -1217,6 +1219,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>> >>       status = check_slot_seqid(seq->seqid, slot);
>> >>       if (status == nfserr_replay_cache) {
>> >>               cstate->slot = slot;
>> >> +             /* Return the cached reply status and set cstate->status
>> >> +              * for nfsd4_svc_encode_compoundres processing*/
>> >> +             status = nfsd4_replay_cache_entry(resp);
>> >> +             cstate->status = nfserr_replay_cache;
>> >>               goto replay_cache;
>> >>       }
>> >>       if (status)
>> >> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
>> >> index c6b490e..57afb33 100644
>> >> --- a/fs/nfsd/nfs4xdr.c
>> >> +++ b/fs/nfsd/nfs4xdr.c
>> >> @@ -3059,6 +3059,19 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
>> >>               iov = &rqstp->rq_res.head[0];
>> >>       iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
>> >>       BUG_ON(iov->iov_len > PAGE_SIZE);
>> >> +#ifdef CONFIG_NFSD_V4_1
>> >> +     if (resp->cstate.slot != NULL) {
>> >> +             if (resp->cstate.status == nfserr_replay_cache) {
>> >> +                     iov->iov_len = resp->cstate.iovlen;
>> >> +             } else {
>> >> +                     nfsd4_set_cache_entry(resp);
>> >> +                     dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
>> >> +                     resp->cstate.slot->sl_inuse = 0;
>> >> +             }
>> >> +             if (resp->cstate.slot->sl_session)
>> >
>> > There's no way that sl_session could ever be NULL, so this check is
>> > unneeded.
>>
>> There should have been a comment.
>> CREATE_SESSION sets cstate->slot using the nfs4_client->cl_slot, a
>> single slot which it uses for the CREATE_SESSION replay cache. It does
>> not have a session.
>
> Hm.  In that case, there's no reference count on the client that holds
> the cl_slot structure, so is there anything preventing the cl_slot from
> being freed out from under us  by a concurrent expire_client()?

That is true, we don't bump the client cl_count....

>
> If that's correct: I'm not sure how to solve that problem with a minimum
> of special-casing.

Hmm. I see what you mean. We would like to take a reference to the
nfs4_client in nfsd4_create_session and drop it in
nfs4svc_encode_compoundres  if the session pointer is NULL.

if (resp->cstate->session)
        nfsd4_put_session(resp>cstate->session)
else
       nfsd4_put_client(???)

but - we currently have no nfs4_client back pointer (except in the
session which is null.......)


-->Andy

>
> --b.
>
>>
>>
>> >
>> > While we're at it: let's just eliminate the sl_session pointer from the
>> > slot.  I think all that would be needed would be a pointer to the
>> > session from the cstate, in addition to the pointer to the slot.
>>
>> OK.
>>
>> >
>> > --b.
>> >
>> >> +                     nfsd4_put_session(resp->cstate.slot->sl_session);
>> >> +     }
>> >> +#endif /* CONFIG_NFSD_V4_1 */
>> >>       return 1;
>> >>  }
>> >>
>> >> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
>> >> index cde8947..5c0d376 100644
>> >> --- a/include/linux/nfsd/xdr4.h
>> >> +++ b/include/linux/nfsd/xdr4.h
>> >> @@ -51,6 +51,7 @@ struct nfsd4_compound_state {
>> >>       /* For sessions DRC */
>> >>       struct nfsd4_slot       *slot;
>> >>       __be32                  *statp;
>> >> +     size_t                  iovlen;
>> >>       u32                     status;
>> >>  };
>> >>
>> >> --
>> >> 1.6.2.1
>> >>
>> > _______________________________________________
>> > pNFS mailing list
>> > pNFS@linux-nfs.org
>> > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
>> >
>

[PATCH v2 22/47] nfsd41: clear DRC cache on free_session

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 07c869d..37865c9 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -442,12 +442,19 @@ release_session(struct nfsd4_session *ses)
 	nfsd4_put_session(ses);
 }
 
+static void nfsd4_release_respages(struct page **respages, short resused);
+
 void
 free_session(struct kref *kref)
 {
 	struct nfsd4_session *ses;
+	int i;
 
 	ses = container_of(kref, struct nfsd4_session, se_ref);
+	for (i = 0; i < ses->se_fnumslots; i++) {
+		struct nfsd4_cache_entry *e = &ses->se_slots[i].sl_cache_entry;
+		nfsd4_release_respages(e->ce_respages, e->ce_resused);
+	}
 	kfree(ses->se_slots);
 	kfree(ses);
 }
-- 
1.6.2.1

[PATCH v2 23/47] nfsd41: create_session operation

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the create_session operation confoming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

Look up the client id (generated by the server on exchange_id,
given by the client on create_session).
If neither a confirmed or unconfirmed client is found
then the client id is stale
If a confirmed cilent is found (i.e. we already received
create_session for it) then compare the sequence id
to determine if it's a replay or possibly a mis-ordered rpc.
If the seqid is in order, update the confirmed client seqid
and procedd with updating the session parameters.

If an unconfirmed client_id is found then verify the creds
and seqid.  If both match move the client id to confirmed state
and proceed with processing the create_session.

Currently, we do not support persistent sessions, and RDMA.

alloc_init_session generates a new sessionid and creates
a session structure.

NFSD_PAGES_PER_SLOT is used for the max response cached calculation, and for
the counting of DRC pages using the hard limits set in struct srv_serv.

A note on NFSD_PAGES_PER_SLOT:

Other patches in this series allow for NFSD_PAGES_PER_SLOT + 1 pages to be
cached in a DRC slot when the response size is less than NFSD_PAGES_PER_SLOT *
PAGE_SIZE but xdr_buf pages are used. e.g. a READDIR operation will encode a
small amount of data in the xdr_buf head, and then the READDIR in the xdr_buf
pages.  So, the hard limit calculation use of pages by a session is
underestimated by the number of cached operations using the xdr_buf pages.

Yet another patch caches no pages for the solo sequence operation, or any
compound where cache_this is False.  So the hard limit calculation use of
pages by a session is overestimated by the number of these operations in the
cache.

TODO: improve resource pre-allocation and negotiate session
parameters accordingly.  Respect and possibly adjust
backchannel attributes.

Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
Signed-off-by: Dean Hildebrand <dhildeb@us.ibm.com>
[nfsd41: remove headerpadsz from channel attributes]
Our client and server only support a headerpadsz of 0.
[nfsd41: use DRC limits in fore channel init]
[nfsd41: do not change CREATE_SESSION back channel attrs]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[use sessionid_lock spin lock]
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |  197 +++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c          |  147 ++++++++++++++++++++++++++++++++-
 include/linux/nfsd/state.h |    7 ++
 include/linux/nfsd/xdr4.h  |   21 +++++-
 4 files changed, 368 insertions(+), 4 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 37865c9..e4e2c19 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -68,6 +68,9 @@ static u32 current_delegid = 1;
 static u32 nfs4_init;
 static stateid_t zerostateid;             /* bits all 0 */
 static stateid_t onestateid;              /* bits all 1 */
+#ifdef CONFIG_NFSD_V4_1
+static u64 current_sessionid = 1;
+#endif /* CONFIG_NFSD_V4_1 */
 
 #define ZERO_STATEID(stateid) (!memcmp((stateid), &zerostateid, sizeof(stateid_t)))
 #define ONE_STATEID(stateid)  (!memcmp((stateid), &onestateid, sizeof(stateid_t)))
@@ -402,6 +405,138 @@ dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
 	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
 }
 
+static void
+gen_sessionid(struct nfsd4_session *ses)
+{
+	struct nfs4_client *clp = ses->se_client;
+	struct nfsd4_sessionid *sid;
+
+	sid = (struct nfsd4_sessionid *)ses->se_sessionid.data;
+	sid->clientid = clp->cl_clientid;
+	sid->sequence = current_sessionid++;
+	sid->reserved = 0;
+}
+
+/*
+ * Give the client the number of slots it requests bound by
+ * NFSD_MAX_SLOTS_PER_SESSION and by sv_drc_max_pages.
+ *
+ * If we run out of pages (sv_drc_pages_used == sv_drc_max_pages) we
+ * should (up to a point) re-negotiate active sessions and reduce their
+ * slot usage to make rooom for new connections. For now we just fail the
+ * create session.
+ */
+static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
+{
+	int status = 0, np = fchan->maxreqs * NFSD_PAGES_PER_SLOT;
+
+	spin_lock(&nfsd_serv->sv_lock);
+	if (np + nfsd_serv->sv_drc_pages_used > nfsd_serv->sv_drc_max_pages)
+		np = nfsd_serv->sv_drc_max_pages - nfsd_serv->sv_drc_pages_used;
+	nfsd_serv->sv_drc_pages_used += np;
+	spin_unlock(&nfsd_serv->sv_lock);
+
+	if (np <= 0) {
+		status = nfserr_resource;
+		fchan->maxreqs = 0;
+	} else
+		fchan->maxreqs = np / NFSD_PAGES_PER_SLOT;
+
+	return status;
+}
+
+/*
+ * fchan holds the client values on input, and the server values on output
+ */
+static int init_forechannel_attrs(struct svc_rqst *rqstp,
+				    struct nfsd4_session *session,
+				    struct nfsd4_channel_attrs *fchan)
+{
+	int status = 0;
+	__u32   maxcount = svc_max_payload(rqstp);
+
+	/* headerpadsz set to zero in encode routine*/
+
+	/* Use the client's max request and max response size if possible */
+	if (fchan->maxreq_sz > maxcount)
+		fchan->maxreq_sz = maxcount;
+	session->se_fmaxreq_sz = fchan->maxreq_sz;
+
+	if (fchan->maxresp_sz > maxcount)
+		fchan->maxresp_sz = maxcount;
+	session->se_fmaxresp_sz = fchan->maxresp_sz;
+
+	/* Set the max response cached size our default which is
+	 * a multiple of PAGE_SIZE and small */
+	session->se_fmaxresp_cached = NFSD_PAGES_PER_SLOT * PAGE_SIZE;
+	fchan->maxresp_cached = session->se_fmaxresp_cached;
+
+	/* Use the client's maxops if possible */
+	if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND)
+		fchan->maxops = NFSD_MAX_OPS_PER_COMPOUND;
+	session->se_fmaxops = fchan->maxops;
+
+	/* try to use the client requested number of slots */
+	if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
+		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
+
+	/* FIXME: Error means no more DRC pages so the server should
+	 * recover pages from existing sessions. For now fail session
+	 * creation.
+	 */
+	status = set_forechannel_maxreqs(fchan);
+
+	session->se_fnumslots = fchan->maxreqs;
+	return status;
+}
+
+static int
+alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
+		   struct nfsd4_create_session *cses)
+{
+	struct nfsd4_session *new;
+	int idx, status = nfserr_resource, slotsize, i;
+
+	new = kzalloc(sizeof(*new), GFP_KERNEL);
+	if (!new)
+		goto out;
+
+	/* FIXME: For now, we just accept the client back channel attributes. */
+	status = init_forechannel_attrs(rqstp, new, &cses->fore_channel);
+	if (status)
+		goto out_free;
+
+	slotsize = new->se_fnumslots * sizeof(struct nfsd4_slot);
+	new->se_slots = kzalloc(slotsize, GFP_KERNEL);
+	if (!new->se_slots)
+		goto out_free;
+
+	for (i = 0; i < new->se_fnumslots; i++)
+		new->se_slots[i].sl_session = new;
+
+	new->se_client = clp;
+	gen_sessionid(new);
+	idx = hash_sessionid(&new->se_sessionid);
+	memcpy(clp->cl_sessionid.data, new->se_sessionid.data,
+	       NFS4_MAX_SESSIONID_LEN);
+
+	new->se_flags = cses->flags;
+	kref_init(&new->se_ref);
+	INIT_LIST_HEAD(&new->se_hash);
+	INIT_LIST_HEAD(&new->se_perclnt);
+	spin_lock(&sessionid_lock);
+	list_add(&new->se_hash, &sessionid_hashtbl[idx]);
+	list_add(&new->se_perclnt, &clp->cl_sessions);
+	spin_unlock(&sessionid_lock);
+
+	status = nfs_ok;
+out:
+	return status;
+out_free:
+	kfree(new);
+	goto out;
+}
+
 /* caller must hold sessionid_lock */
 static struct nfsd4_session *
 find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
@@ -1186,7 +1321,67 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		     struct nfsd4_compound_state *cstate,
 		     struct nfsd4_create_session *cr_ses)
 {
-	return -1;	/* stub */
+	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
+	struct nfs4_client *conf, *unconf;
+	int status = 0;
+
+	nfs4_lock_state();
+	unconf = find_unconfirmed_client(&cr_ses->clientid);
+	conf = find_confirmed_client(&cr_ses->clientid);
+
+	if (conf) {
+		status = nfs_ok;
+		if (conf->cl_seqid == cr_ses->seqid) {
+			dprintk("Got a create_session replay! seqid= %d\n",
+				conf->cl_seqid);
+			goto out_replay;
+		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
+			status = nfserr_seq_misordered;
+			dprintk("Sequence misordered!\n");
+			dprintk("Expected seqid= %d but got seqid= %d\n",
+				conf->cl_seqid, cr_ses->seqid);
+			goto out;
+		}
+		conf->cl_seqid++;
+	} else if (unconf) {
+		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
+		    (ip_addr != unconf->cl_addr)) {
+			status = nfserr_clid_inuse;
+			goto out;
+		}
+
+		if (unconf->cl_seqid != cr_ses->seqid) {
+			status = nfserr_seq_misordered;
+			goto out;
+		}
+
+		move_to_confirmed(unconf);
+
+		/*
+		 * We do not support RDMA or persistent sessions
+		 */
+		cr_ses->flags &= ~SESSION4_PERSIST;
+		cr_ses->flags &= ~SESSION4_RDMA;
+
+		conf = unconf;
+	} else {
+		status = nfserr_stale_clientid;
+		goto out;
+	}
+
+	status = alloc_init_session(rqstp, conf, cr_ses);
+	if (status)
+		goto out;
+
+out_replay:
+	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
+	       NFS4_MAX_SESSIONID_LEN);
+	cr_ses->seqid = conf->cl_seqid;
+
+out:
+	nfs4_unlock_state();
+	dprintk("%s returns %d\n", __func__, ntohl(status));
+	return status;
 }
 
 __be32
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 57afb33..60db854 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1100,7 +1100,108 @@ static __be32
 nfsd4_decode_create_session(struct nfsd4_compoundargs *argp,
 			    struct nfsd4_create_session *sess)
 {
-	return nfserr_opnotsupp;	/* stub */
+	DECODE_HEAD;
+
+	u32 dummy;
+	char *machine_name;
+	int i;
+	int nr_secflavs;
+
+	READ_BUF(16);
+	COPYMEM(&sess->clientid, 8);
+	READ32(sess->seqid);
+	READ32(sess->flags);
+
+	/* Fore channel attrs */
+	READ_BUF(28);
+	READ32(dummy); /* headerpadsz is always 0 */
+	READ32(sess->fore_channel.maxreq_sz);
+	READ32(sess->fore_channel.maxresp_sz);
+	READ32(sess->fore_channel.maxresp_cached);
+	READ32(sess->fore_channel.maxops);
+	READ32(sess->fore_channel.maxreqs);
+	READ32(sess->fore_channel.nr_rdma_attrs);
+	if (sess->fore_channel.nr_rdma_attrs == 1) {
+		READ_BUF(4);
+		READ32(sess->fore_channel.rdma_attrs);
+	} else if (sess->fore_channel.nr_rdma_attrs > 1) {
+		dprintk("Too many fore channel attr bitmaps!\n");
+		goto xdr_error;
+	}
+
+	/* Back channel attrs */
+	READ_BUF(28);
+	READ32(dummy); /* headerpadsz is always 0 */
+	READ32(sess->back_channel.maxreq_sz);
+	READ32(sess->back_channel.maxresp_sz);
+	READ32(sess->back_channel.maxresp_cached);
+	READ32(sess->back_channel.maxops);
+	READ32(sess->back_channel.maxreqs);
+	READ32(sess->back_channel.nr_rdma_attrs);
+	if (sess->back_channel.nr_rdma_attrs == 1) {
+		READ_BUF(4);
+		READ32(sess->back_channel.rdma_attrs);
+	} else if (sess->back_channel.nr_rdma_attrs > 1) {
+		dprintk("Too many back channel attr bitmaps!\n");
+		goto xdr_error;
+	}
+
+	READ_BUF(8);
+	READ32(sess->callback_prog);
+
+	/* callback_sec_params4 */
+	READ32(nr_secflavs);
+	for (i = 0; i < nr_secflavs; ++i) {
+		READ_BUF(4);
+		READ32(dummy);
+		switch (dummy) {
+		case RPC_AUTH_NULL:
+			/* Nothing to read */
+			break;
+		case RPC_AUTH_UNIX:
+			READ_BUF(8);
+			/* stamp */
+			READ32(dummy);
+
+			/* machine name */
+			READ32(dummy);
+			READ_BUF(dummy);
+			SAVEMEM(machine_name, dummy);
+
+			/* uid, gid */
+			READ_BUF(8);
+			READ32(sess->uid);
+			READ32(sess->gid);
+
+			/* more gids */
+			READ_BUF(4);
+			READ32(dummy);
+			READ_BUF(dummy * 4);
+			for (i = 0; i < dummy; ++i)
+				READ32(dummy);
+			break;
+		case RPC_AUTH_GSS:
+			dprintk("RPC_AUTH_GSS callback secflavor "
+				"not supported!\n");
+			READ_BUF(8);
+			/* gcbp_service */
+			READ32(dummy);
+			/* gcbp_handle_from_server */
+			READ32(dummy);
+			READ_BUF(dummy);
+			p += XDR_QUADLEN(dummy);
+			/* gcbp_handle_from_client */
+			READ_BUF(4);
+			READ32(dummy);
+			READ_BUF(dummy);
+			p += XDR_QUADLEN(dummy);
+			break;
+		default:
+			dprintk("Illegal callback secflavor\n");
+			return nfserr_inval;
+		}
+	}
+	DECODE_TAIL;
 }
 
 static __be32
@@ -2829,7 +2930,49 @@ static __be32
 nfsd4_encode_create_session(struct nfsd4_compoundres *resp, int nfserr,
 			    struct nfsd4_create_session *sess)
 {
-	/* stub */
+	ENCODE_HEAD;
+
+	if (nfserr)
+		goto out;
+
+	RESERVE_SPACE(24);
+	WRITEMEM(sess->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(sess->seqid);
+	WRITE32(sess->flags);
+	ADJUST_ARGS();
+
+	RESERVE_SPACE(28);
+	WRITE32(0); /* headerpadsz */
+	WRITE32(sess->fore_channel.maxreq_sz);
+	WRITE32(sess->fore_channel.maxresp_sz);
+	WRITE32(sess->fore_channel.maxresp_cached);
+	WRITE32(sess->fore_channel.maxops);
+	WRITE32(sess->fore_channel.maxreqs);
+	WRITE32(sess->fore_channel.nr_rdma_attrs);
+	ADJUST_ARGS();
+
+	if (sess->fore_channel.nr_rdma_attrs) {
+		RESERVE_SPACE(4);
+		WRITE32(sess->fore_channel.rdma_attrs);
+		ADJUST_ARGS();
+	}
+
+	RESERVE_SPACE(28);
+	WRITE32(0); /* headerpadsz */
+	WRITE32(sess->back_channel.maxreq_sz);
+	WRITE32(sess->back_channel.maxresp_sz);
+	WRITE32(sess->back_channel.maxresp_cached);
+	WRITE32(sess->back_channel.maxops);
+	WRITE32(sess->back_channel.maxreqs);
+	WRITE32(sess->back_channel.nr_rdma_attrs);
+	ADJUST_ARGS();
+
+	if (sess->back_channel.nr_rdma_attrs) {
+		RESERVE_SPACE(4);
+		WRITE32(sess->back_channel.rdma_attrs);
+		ADJUST_ARGS();
+	}
+out:
 	return nfserr;
 }
 
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 8ca6a82..98d7b1c 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -99,8 +99,12 @@ struct nfs4_callback {
 	struct rpc_clnt *       cb_client;
 };
 
+/* Maximum number of slots per session. 128 is useful for long haul TCP */
+#define NFSD_MAX_SLOTS_PER_SESSION	128
 /* Maximum number of pages per slot cache entry */
 #define NFSD_PAGES_PER_SLOT	1
+/* Maximum number of operations per session compound */
+#define NFSD_MAX_OPS_PER_COMPOUND	16
 
 struct nfsd4_cache_entry {
 	__be32		ce_status;
@@ -188,6 +192,9 @@ struct nfs4_client {
 	struct list_head	cl_sessions;
 	u32			cl_seqid;       /* seqid for create_session */
 	u32			cl_exchange_flags;
+	struct nfs4_sessionid	cl_sessionid;
+
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 5c0d376..c7bf0a1 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -360,8 +360,27 @@ struct nfsd4_exchange_id {
 	int		spa_how;
 };
 
+struct nfsd4_channel_attrs {
+	u32		headerpadsz;
+	u32		maxreq_sz;
+	u32		maxresp_sz;
+	u32		maxresp_cached;
+	u32		maxops;
+	u32		maxreqs;
+	u32		nr_rdma_attrs;
+	u32		rdma_attrs;
+};
+
 struct nfsd4_create_session {
-	int	foo;	/* stub */
+	clientid_t		clientid;
+	struct nfs4_sessionid	sessionid;
+	u32			seqid;
+	u32			flags;
+	struct nfsd4_channel_attrs fore_channel;
+	struct nfsd4_channel_attrs back_channel;
+	u32			callback_prog;
+	u32			uid;
+	u32			gid;
 };
 
 struct nfsd4_sequence {
-- 
1.6.2.1

Re: [PATCH v2 23/47] nfsd41: create_session operation

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:57AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Implement the create_session operation confoming to
> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26
> 
> Look up the client id (generated by the server on exchange_id,
> given by the client on create_session).
> If neither a confirmed or unconfirmed client is found
> then the client id is stale
> If a confirmed cilent is found (i.e. we already received
> create_session for it) then compare the sequence id
> to determine if it's a replay or possibly a mis-ordered rpc.
> If the seqid is in order, update the confirmed client seqid
> and procedd with updating the session parameters.
> 
> If an unconfirmed client_id is found then verify the creds
> and seqid.  If both match move the client id to confirmed state
> and proceed with processing the create_session.

The above two paragraphs just summarize the code step-by-step; that
doesn't belong in the commit message.

> 
> Currently, we do not support persistent sessions, and RDMA.
> 
> alloc_init_session generates a new sessionid and creates
> a session structure.
> 
> NFSD_PAGES_PER_SLOT is used for the max response cached calculation, and for
> the counting of DRC pages using the hard limits set in struct srv_serv.
> 
> A note on NFSD_PAGES_PER_SLOT:
> 
> Other patches in this series allow for NFSD_PAGES_PER_SLOT + 1 pages to be
> cached in a DRC slot when the response size is less than NFSD_PAGES_PER_SLOT *
> PAGE_SIZE but xdr_buf pages are used. e.g. a READDIR operation will encode a
> small amount of data in the xdr_buf head, and then the READDIR in the xdr_buf
> pages.  So, the hard limit calculation use of pages by a session is
> underestimated by the number of cached operations using the xdr_buf pages.

I think this might all be clearer if we started with a #define for the
maximum cached response size, and then calculated NFSD_PAGES_PER_SLOT
from that.  Also, factor in the +1 (or +3, or whatever it should be)
into the PAGES_PER_SLOT so we don't have to remember to do that
everywhere.

> Yet another patch caches no pages for the solo sequence operation, or any
> compound where cache_this is False.  So the hard limit calculation use of
> pages by a session is overestimated by the number of these operations in the
> cache.
> 
> TODO: improve resource pre-allocation and negotiate session
> parameters accordingly.  Respect and possibly adjust
> backchannel attributes.
> 
> Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
> Signed-off-by: Dean Hildebrand <dhildeb@us.ibm.com>
> [nfsd41: remove headerpadsz from channel attributes]
> Our client and server only support a headerpadsz of 0.
> [nfsd41: use DRC limits in fore channel init]
> [nfsd41: do not change CREATE_SESSION back channel attrs]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [use sessionid_lock spin lock]
> [nfsd41: use bool inuse for slot state]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |  197 +++++++++++++++++++++++++++++++++++++++++++-
>  fs/nfsd/nfs4xdr.c          |  147 ++++++++++++++++++++++++++++++++-
>  include/linux/nfsd/state.h |    7 ++
>  include/linux/nfsd/xdr4.h  |   21 +++++-
>  4 files changed, 368 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 37865c9..e4e2c19 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -68,6 +68,9 @@ static u32 current_delegid = 1;
>  static u32 nfs4_init;
>  static stateid_t zerostateid;             /* bits all 0 */
>  static stateid_t onestateid;              /* bits all 1 */
> +#ifdef CONFIG_NFSD_V4_1
> +static u64 current_sessionid = 1;
> +#endif /* CONFIG_NFSD_V4_1 */
>  
>  #define ZERO_STATEID(stateid) (!memcmp((stateid), &zerostateid, sizeof(stateid_t)))
>  #define ONE_STATEID(stateid)  (!memcmp((stateid), &onestateid, sizeof(stateid_t)))
> @@ -402,6 +405,138 @@ dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
>  	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
>  }
>  
> +static void
> +gen_sessionid(struct nfsd4_session *ses)
> +{
> +	struct nfs4_client *clp = ses->se_client;
> +	struct nfsd4_sessionid *sid;
> +
> +	sid = (struct nfsd4_sessionid *)ses->se_sessionid.data;
> +	sid->clientid = clp->cl_clientid;
> +	sid->sequence = current_sessionid++;
> +	sid->reserved = 0;
> +}
> +
> +/*
> + * Give the client the number of slots it requests bound by
> + * NFSD_MAX_SLOTS_PER_SESSION and by sv_drc_max_pages.
> + *
> + * If we run out of pages (sv_drc_pages_used == sv_drc_max_pages) we
> + * should (up to a point) re-negotiate active sessions and reduce their
> + * slot usage to make rooom for new connections. For now we just fail the
> + * create session.
> + */
> +static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
> +{
> +	int status = 0, np = fchan->maxreqs * NFSD_PAGES_PER_SLOT;
> +
> +	spin_lock(&nfsd_serv->sv_lock);
> +	if (np + nfsd_serv->sv_drc_pages_used > nfsd_serv->sv_drc_max_pages)
> +		np = nfsd_serv->sv_drc_max_pages - nfsd_serv->sv_drc_pages_used;
> +	nfsd_serv->sv_drc_pages_used += np;
> +	spin_unlock(&nfsd_serv->sv_lock);

Don't use the sv_lock for this.

(I greatly appreciate any help with the locking, but the difficult part
isn't adding lock calls, but developing and explaining clear rules about
what locks are taken where, and why.)

> +
> +	if (np <= 0) {

It's too late--we've already increased sv_drc_pages_used.  This check
needs to happen earlier.

> +		status = nfserr_resource;
> +		fchan->maxreqs = 0;
> +	} else
> +		fchan->maxreqs = np / NFSD_PAGES_PER_SLOT;

Check to make sure that's not zero?

> +
> +	return status;
> +}
> +
> +/*
> + * fchan holds the client values on input, and the server values on output
> + */
> +static int init_forechannel_attrs(struct svc_rqst *rqstp,
> +				    struct nfsd4_session *session,
> +				    struct nfsd4_channel_attrs *fchan)
> +{
> +	int status = 0;
> +	__u32   maxcount = svc_max_payload(rqstp);
> +
> +	/* headerpadsz set to zero in encode routine*/

Space after "routine".

> +
> +	/* Use the client's max request and max response size if possible */
> +	if (fchan->maxreq_sz > maxcount)
> +		fchan->maxreq_sz = maxcount;
> +	session->se_fmaxreq_sz = fchan->maxreq_sz;
> +
> +	if (fchan->maxresp_sz > maxcount)
> +		fchan->maxresp_sz = maxcount;
> +	session->se_fmaxresp_sz = fchan->maxresp_sz;
> +
> +	/* Set the max response cached size our default which is
> +	 * a multiple of PAGE_SIZE and small */
> +	session->se_fmaxresp_cached = NFSD_PAGES_PER_SLOT * PAGE_SIZE;
> +	fchan->maxresp_cached = session->se_fmaxresp_cached;
> +
> +	/* Use the client's maxops if possible */
> +	if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND)
> +		fchan->maxops = NFSD_MAX_OPS_PER_COMPOUND;
> +	session->se_fmaxops = fchan->maxops;
> +
> +	/* try to use the client requested number of slots */
> +	if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
> +		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
> +
> +	/* FIXME: Error means no more DRC pages so the server should
> +	 * recover pages from existing sessions. For now fail session
> +	 * creation.
> +	 */
> +	status = set_forechannel_maxreqs(fchan);
> +
> +	session->se_fnumslots = fchan->maxreqs;
> +	return status;
> +}
> +
> +static int
> +alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
> +		   struct nfsd4_create_session *cses)
> +{
> +	struct nfsd4_session *new;
> +	int idx, status = nfserr_resource, slotsize, i;
> +
> +	new = kzalloc(sizeof(*new), GFP_KERNEL);
> +	if (!new)
> +		goto out;

Just return.

> +
> +	/* FIXME: For now, we just accept the client back channel attributes. */
> +	status = init_forechannel_attrs(rqstp, new, &cses->fore_channel);
> +	if (status)
> +		goto out_free;
> +
> +	slotsize = new->se_fnumslots * sizeof(struct nfsd4_slot);
> +	new->se_slots = kzalloc(slotsize, GFP_KERNEL);
> +	if (!new->se_slots)
> +		goto out_free;

Let's just allocate the session and slot table as one chunk of size
(sizeof(struct nfsd4_session)+ numslots * sizeof(struct nfsd4_slot).

> +
> +	for (i = 0; i < new->se_fnumslots; i++)
> +		new->se_slots[i].sl_session = new;

And let's get rid of sl_session.

> +
> +	new->se_client = clp;
> +	gen_sessionid(new);
> +	idx = hash_sessionid(&new->se_sessionid);
> +	memcpy(clp->cl_sessionid.data, new->se_sessionid.data,
> +	       NFS4_MAX_SESSIONID_LEN);
> +
> +	new->se_flags = cses->flags;
> +	kref_init(&new->se_ref);
> +	INIT_LIST_HEAD(&new->se_hash);
> +	INIT_LIST_HEAD(&new->se_perclnt);

These INIT_LIST_HEAD's are redundant given the immediately following
list_add's.

> +	spin_lock(&sessionid_lock);
> +	list_add(&new->se_hash, &sessionid_hashtbl[idx]);
> +	list_add(&new->se_perclnt, &clp->cl_sessions);
> +	spin_unlock(&sessionid_lock);
> +
> +	status = nfs_ok;
> +out:
> +	return status;
> +out_free:
> +	kfree(new);
> +	goto out;
> +}
> +
>  /* caller must hold sessionid_lock */
>  static struct nfsd4_session *
>  find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
> @@ -1186,7 +1321,67 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  		     struct nfsd4_compound_state *cstate,
>  		     struct nfsd4_create_session *cr_ses)
>  {
> -	return -1;	/* stub */
> +	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
> +	struct nfs4_client *conf, *unconf;
> +	int status = 0;
> +
> +	nfs4_lock_state();
> +	unconf = find_unconfirmed_client(&cr_ses->clientid);
> +	conf = find_confirmed_client(&cr_ses->clientid);
> +
> +	if (conf) {
> +		status = nfs_ok;
> +		if (conf->cl_seqid == cr_ses->seqid) {
> +			dprintk("Got a create_session replay! seqid= %d\n",
> +				conf->cl_seqid);
> +			goto out_replay;
> +		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
> +			status = nfserr_seq_misordered;
> +			dprintk("Sequence misordered!\n");
> +			dprintk("Expected seqid= %d but got seqid= %d\n",
> +				conf->cl_seqid, cr_ses->seqid);
> +			goto out;
> +		}
> +		conf->cl_seqid++;
> +	} else if (unconf) {
> +		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
> +		    (ip_addr != unconf->cl_addr)) {
> +			status = nfserr_clid_inuse;
> +			goto out;
> +		}
> +
> +		if (unconf->cl_seqid != cr_ses->seqid) {
> +			status = nfserr_seq_misordered;
> +			goto out;
> +		}
> +
> +		move_to_confirmed(unconf);
> +
> +		/*
> +		 * We do not support RDMA or persistent sessions
> +		 */
> +		cr_ses->flags &= ~SESSION4_PERSIST;
> +		cr_ses->flags &= ~SESSION4_RDMA;
> +
> +		conf = unconf;
> +	} else {
> +		status = nfserr_stale_clientid;
> +		goto out;
> +	}
> +
> +	status = alloc_init_session(rqstp, conf, cr_ses);
> +	if (status)
> +		goto out;
> +
> +out_replay:
> +	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
> +	       NFS4_MAX_SESSIONID_LEN);
> +	cr_ses->seqid = conf->cl_seqid;
> +
> +out:
> +	nfs4_unlock_state();
> +	dprintk("%s returns %d\n", __func__, ntohl(status));
> +	return status;
>  }
>  
>  __be32
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index 57afb33..60db854 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -1100,7 +1100,108 @@ static __be32
>  nfsd4_decode_create_session(struct nfsd4_compoundargs *argp,
>  			    struct nfsd4_create_session *sess)
>  {
> -	return nfserr_opnotsupp;	/* stub */
> +	DECODE_HEAD;
> +
> +	u32 dummy;
> +	char *machine_name;
> +	int i;
> +	int nr_secflavs;
> +
> +	READ_BUF(16);
> +	COPYMEM(&sess->clientid, 8);
> +	READ32(sess->seqid);
> +	READ32(sess->flags);
> +
> +	/* Fore channel attrs */
> +	READ_BUF(28);
> +	READ32(dummy); /* headerpadsz is always 0 */
> +	READ32(sess->fore_channel.maxreq_sz);
> +	READ32(sess->fore_channel.maxresp_sz);
> +	READ32(sess->fore_channel.maxresp_cached);
> +	READ32(sess->fore_channel.maxops);
> +	READ32(sess->fore_channel.maxreqs);
> +	READ32(sess->fore_channel.nr_rdma_attrs);
> +	if (sess->fore_channel.nr_rdma_attrs == 1) {
> +		READ_BUF(4);
> +		READ32(sess->fore_channel.rdma_attrs);
> +	} else if (sess->fore_channel.nr_rdma_attrs > 1) {
> +		dprintk("Too many fore channel attr bitmaps!\n");
> +		goto xdr_error;
> +	}
> +
> +	/* Back channel attrs */
> +	READ_BUF(28);
> +	READ32(dummy); /* headerpadsz is always 0 */
> +	READ32(sess->back_channel.maxreq_sz);
> +	READ32(sess->back_channel.maxresp_sz);
> +	READ32(sess->back_channel.maxresp_cached);
> +	READ32(sess->back_channel.maxops);
> +	READ32(sess->back_channel.maxreqs);
> +	READ32(sess->back_channel.nr_rdma_attrs);
> +	if (sess->back_channel.nr_rdma_attrs == 1) {
> +		READ_BUF(4);
> +		READ32(sess->back_channel.rdma_attrs);
> +	} else if (sess->back_channel.nr_rdma_attrs > 1) {
> +		dprintk("Too many back channel attr bitmaps!\n");
> +		goto xdr_error;
> +	}
> +
> +	READ_BUF(8);
> +	READ32(sess->callback_prog);
> +
> +	/* callback_sec_params4 */
> +	READ32(nr_secflavs);
> +	for (i = 0; i < nr_secflavs; ++i) {
> +		READ_BUF(4);
> +		READ32(dummy);
> +		switch (dummy) {
> +		case RPC_AUTH_NULL:
> +			/* Nothing to read */
> +			break;
> +		case RPC_AUTH_UNIX:
> +			READ_BUF(8);
> +			/* stamp */
> +			READ32(dummy);
> +
> +			/* machine name */
> +			READ32(dummy);
> +			READ_BUF(dummy);
> +			SAVEMEM(machine_name, dummy);
> +
> +			/* uid, gid */
> +			READ_BUF(8);
> +			READ32(sess->uid);
> +			READ32(sess->gid);
> +
> +			/* more gids */
> +			READ_BUF(4);
> +			READ32(dummy);
> +			READ_BUF(dummy * 4);
> +			for (i = 0; i < dummy; ++i)
> +				READ32(dummy);
> +			break;
> +		case RPC_AUTH_GSS:
> +			dprintk("RPC_AUTH_GSS callback secflavor "
> +				"not supported!\n");
> +			READ_BUF(8);
> +			/* gcbp_service */
> +			READ32(dummy);
> +			/* gcbp_handle_from_server */
> +			READ32(dummy);
> +			READ_BUF(dummy);
> +			p += XDR_QUADLEN(dummy);
> +			/* gcbp_handle_from_client */
> +			READ_BUF(4);
> +			READ32(dummy);
> +			READ_BUF(dummy);
> +			p += XDR_QUADLEN(dummy);
> +			break;
> +		default:
> +			dprintk("Illegal callback secflavor\n");
> +			return nfserr_inval;
> +		}
> +	}
> +	DECODE_TAIL;
>  }
>  
>  static __be32
> @@ -2829,7 +2930,49 @@ static __be32
>  nfsd4_encode_create_session(struct nfsd4_compoundres *resp, int nfserr,
>  			    struct nfsd4_create_session *sess)
>  {
> -	/* stub */
> +	ENCODE_HEAD;
> +
> +	if (nfserr)
> +		goto out;
> +
> +	RESERVE_SPACE(24);
> +	WRITEMEM(sess->sessionid.data, NFS4_MAX_SESSIONID_LEN);
> +	WRITE32(sess->seqid);
> +	WRITE32(sess->flags);
> +	ADJUST_ARGS();
> +
> +	RESERVE_SPACE(28);
> +	WRITE32(0); /* headerpadsz */
> +	WRITE32(sess->fore_channel.maxreq_sz);
> +	WRITE32(sess->fore_channel.maxresp_sz);
> +	WRITE32(sess->fore_channel.maxresp_cached);
> +	WRITE32(sess->fore_channel.maxops);
> +	WRITE32(sess->fore_channel.maxreqs);
> +	WRITE32(sess->fore_channel.nr_rdma_attrs);
> +	ADJUST_ARGS();
> +
> +	if (sess->fore_channel.nr_rdma_attrs) {
> +		RESERVE_SPACE(4);
> +		WRITE32(sess->fore_channel.rdma_attrs);
> +		ADJUST_ARGS();
> +	}
> +
> +	RESERVE_SPACE(28);
> +	WRITE32(0); /* headerpadsz */
> +	WRITE32(sess->back_channel.maxreq_sz);
> +	WRITE32(sess->back_channel.maxresp_sz);
> +	WRITE32(sess->back_channel.maxresp_cached);
> +	WRITE32(sess->back_channel.maxops);
> +	WRITE32(sess->back_channel.maxreqs);
> +	WRITE32(sess->back_channel.nr_rdma_attrs);
> +	ADJUST_ARGS();
> +
> +	if (sess->back_channel.nr_rdma_attrs) {
> +		RESERVE_SPACE(4);
> +		WRITE32(sess->back_channel.rdma_attrs);
> +		ADJUST_ARGS();
> +	}
> +out:
>  	return nfserr;
>  }
>  
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 8ca6a82..98d7b1c 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -99,8 +99,12 @@ struct nfs4_callback {
>  	struct rpc_clnt *       cb_client;
>  };
>  
> +/* Maximum number of slots per session. 128 is useful for long haul TCP */
> +#define NFSD_MAX_SLOTS_PER_SESSION	128
>  /* Maximum number of pages per slot cache entry */
>  #define NFSD_PAGES_PER_SLOT	1
> +/* Maximum number of operations per session compound */
> +#define NFSD_MAX_OPS_PER_COMPOUND	16
>  
>  struct nfsd4_cache_entry {
>  	__be32		ce_status;
> @@ -188,6 +192,9 @@ struct nfs4_client {
>  	struct list_head	cl_sessions;
>  	u32			cl_seqid;       /* seqid for create_session */
>  	u32			cl_exchange_flags;
> +	struct nfs4_sessionid	cl_sessionid;
> +
> +	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
>  #endif /* CONFIG_NFSD_V4_1 */
>  };
>  
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 5c0d376..c7bf0a1 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -360,8 +360,27 @@ struct nfsd4_exchange_id {
>  	int		spa_how;
>  };
>  
> +struct nfsd4_channel_attrs {
> +	u32		headerpadsz;
> +	u32		maxreq_sz;
> +	u32		maxresp_sz;
> +	u32		maxresp_cached;
> +	u32		maxops;
> +	u32		maxreqs;
> +	u32		nr_rdma_attrs;
> +	u32		rdma_attrs;
> +};
> +
>  struct nfsd4_create_session {
> -	int	foo;	/* stub */
> +	clientid_t		clientid;
> +	struct nfs4_sessionid	sessionid;
> +	u32			seqid;
> +	u32			flags;
> +	struct nfsd4_channel_attrs fore_channel;
> +	struct nfsd4_channel_attrs back_channel;
> +	u32			callback_prog;
> +	u32			uid;
> +	u32			gid;
>  };
>  
>  struct nfsd4_sequence {
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 23/47] nfsd41: create_session operation

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:32:57AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Implement the create_session operation confoming to
> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26
> 
> Look up the client id (generated by the server on exchange_id,
> given by the client on create_session).
> If neither a confirmed or unconfirmed client is found
> then the client id is stale
> If a confirmed cilent is found (i.e. we already received
> create_session for it) then compare the sequence id
> to determine if it's a replay or possibly a mis-ordered rpc.
> If the seqid is in order, update the confirmed client seqid
> and procedd with updating the session parameters.
> 
> If an unconfirmed client_id is found then verify the creds
> and seqid.  If both match move the client id to confirmed state
> and proceed with processing the create_session.
> 
> Currently, we do not support persistent sessions, and RDMA.
> 
> alloc_init_session generates a new sessionid and creates
> a session structure.
> 
> NFSD_PAGES_PER_SLOT is used for the max response cached calculation, and for
> the counting of DRC pages using the hard limits set in struct srv_serv.
> 
> A note on NFSD_PAGES_PER_SLOT:
> 
> Other patches in this series allow for NFSD_PAGES_PER_SLOT + 1 pages to be
> cached in a DRC slot when the response size is less than NFSD_PAGES_PER_SLOT *
> PAGE_SIZE but xdr_buf pages are used. e.g. a READDIR operation will encode a
> small amount of data in the xdr_buf head, and then the READDIR in the xdr_buf
> pages.  So, the hard limit calculation use of pages by a session is
> underestimated by the number of cached operations using the xdr_buf pages.
> 
> Yet another patch caches no pages for the solo sequence operation, or any
> compound where cache_this is False.  So the hard limit calculation use of
> pages by a session is overestimated by the number of these operations in the
> cache.
> 
> TODO: improve resource pre-allocation and negotiate session
> parameters accordingly.  Respect and possibly adjust
> backchannel attributes.
> 
> Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
> Signed-off-by: Dean Hildebrand <dhildeb@us.ibm.com>
> [nfsd41: remove headerpadsz from channel attributes]
> Our client and server only support a headerpadsz of 0.
> [nfsd41: use DRC limits in fore channel init]
> [nfsd41: do not change CREATE_SESSION back channel attrs]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [use sessionid_lock spin lock]
> [nfsd41: use bool inuse for slot state]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |  197 +++++++++++++++++++++++++++++++++++++++++++-
>  fs/nfsd/nfs4xdr.c          |  147 ++++++++++++++++++++++++++++++++-
>  include/linux/nfsd/state.h |    7 ++
>  include/linux/nfsd/xdr4.h  |   21 +++++-
>  4 files changed, 368 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 37865c9..e4e2c19 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -68,6 +68,9 @@ static u32 current_delegid = 1;
>  static u32 nfs4_init;
>  static stateid_t zerostateid;             /* bits all 0 */
>  static stateid_t onestateid;              /* bits all 1 */
> +#ifdef CONFIG_NFSD_V4_1
> +static u64 current_sessionid = 1;
> +#endif /* CONFIG_NFSD_V4_1 */
>  
>  #define ZERO_STATEID(stateid) (!memcmp((stateid), &zerostateid, sizeof(stateid_t)))
>  #define ONE_STATEID(stateid)  (!memcmp((stateid), &onestateid, sizeof(stateid_t)))
> @@ -402,6 +405,138 @@ dump_sessionid(const char *fn, struct nfs4_sessionid *sessionid)
>  	dprintk("%s: %u:%u:%u:%u\n", fn, ptr[0], ptr[1], ptr[2], ptr[3]);
>  }
>  
> +static void
> +gen_sessionid(struct nfsd4_session *ses)
> +{
> +	struct nfs4_client *clp = ses->se_client;
> +	struct nfsd4_sessionid *sid;
> +
> +	sid = (struct nfsd4_sessionid *)ses->se_sessionid.data;
> +	sid->clientid = clp->cl_clientid;
> +	sid->sequence = current_sessionid++;
> +	sid->reserved = 0;
> +}
> +
> +/*
> + * Give the client the number of slots it requests bound by
> + * NFSD_MAX_SLOTS_PER_SESSION and by sv_drc_max_pages.
> + *
> + * If we run out of pages (sv_drc_pages_used == sv_drc_max_pages) we
> + * should (up to a point) re-negotiate active sessions and reduce their
> + * slot usage to make rooom for new connections. For now we just fail the
> + * create session.
> + */
> +static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
> +{
> +	int status = 0, np = fchan->maxreqs * NFSD_PAGES_PER_SLOT;
> +
> +	spin_lock(&nfsd_serv->sv_lock);
> +	if (np + nfsd_serv->sv_drc_pages_used > nfsd_serv->sv_drc_max_pages)
> +		np = nfsd_serv->sv_drc_max_pages - nfsd_serv->sv_drc_pages_used;
> +	nfsd_serv->sv_drc_pages_used += np;
> +	spin_unlock(&nfsd_serv->sv_lock);
> +
> +	if (np <= 0) {
> +		status = nfserr_resource;
> +		fchan->maxreqs = 0;
> +	} else
> +		fchan->maxreqs = np / NFSD_PAGES_PER_SLOT;
> +
> +	return status;
> +}
> +
> +/*
> + * fchan holds the client values on input, and the server values on output
> + */
> +static int init_forechannel_attrs(struct svc_rqst *rqstp,
> +				    struct nfsd4_session *session,
> +				    struct nfsd4_channel_attrs *fchan)
> +{
> +	int status = 0;
> +	__u32   maxcount = svc_max_payload(rqstp);
> +
> +	/* headerpadsz set to zero in encode routine*/
> +
> +	/* Use the client's max request and max response size if possible */
> +	if (fchan->maxreq_sz > maxcount)
> +		fchan->maxreq_sz = maxcount;
> +	session->se_fmaxreq_sz = fchan->maxreq_sz;
> +
> +	if (fchan->maxresp_sz > maxcount)
> +		fchan->maxresp_sz = maxcount;
> +	session->se_fmaxresp_sz = fchan->maxresp_sz;
> +
> +	/* Set the max response cached size our default which is
> +	 * a multiple of PAGE_SIZE and small */
> +	session->se_fmaxresp_cached = NFSD_PAGES_PER_SLOT * PAGE_SIZE;
> +	fchan->maxresp_cached = session->se_fmaxresp_cached;
> +
> +	/* Use the client's maxops if possible */
> +	if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND)
> +		fchan->maxops = NFSD_MAX_OPS_PER_COMPOUND;
> +	session->se_fmaxops = fchan->maxops;
> +
> +	/* try to use the client requested number of slots */
> +	if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
> +		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
> +
> +	/* FIXME: Error means no more DRC pages so the server should
> +	 * recover pages from existing sessions. For now fail session
> +	 * creation.
> +	 */
> +	status = set_forechannel_maxreqs(fchan);
> +
> +	session->se_fnumslots = fchan->maxreqs;
> +	return status;
> +}
> +
> +static int
> +alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
> +		   struct nfsd4_create_session *cses)
> +{
> +	struct nfsd4_session *new;
> +	int idx, status = nfserr_resource, slotsize, i;
> +
> +	new = kzalloc(sizeof(*new), GFP_KERNEL);
> +	if (!new)
> +		goto out;
> +
> +	/* FIXME: For now, we just accept the client back channel attributes. */
> +	status = init_forechannel_attrs(rqstp, new, &cses->fore_channel);
> +	if (status)
> +		goto out_free;
> +
> +	slotsize = new->se_fnumslots * sizeof(struct nfsd4_slot);
> +	new->se_slots = kzalloc(slotsize, GFP_KERNEL);
> +	if (!new->se_slots)
> +		goto out_free;
> +
> +	for (i = 0; i < new->se_fnumslots; i++)
> +		new->se_slots[i].sl_session = new;
> +
> +	new->se_client = clp;
> +	gen_sessionid(new);
> +	idx = hash_sessionid(&new->se_sessionid);
> +	memcpy(clp->cl_sessionid.data, new->se_sessionid.data,
> +	       NFS4_MAX_SESSIONID_LEN);

The sessionid should be part of the new session, not the client, right?

> +
> +	new->se_flags = cses->flags;
> +	kref_init(&new->se_ref);
> +	INIT_LIST_HEAD(&new->se_hash);
> +	INIT_LIST_HEAD(&new->se_perclnt);
> +	spin_lock(&sessionid_lock);
> +	list_add(&new->se_hash, &sessionid_hashtbl[idx]);
> +	list_add(&new->se_perclnt, &clp->cl_sessions);
> +	spin_unlock(&sessionid_lock);
> +
> +	status = nfs_ok;
> +out:
> +	return status;
> +out_free:
> +	kfree(new);
> +	goto out;
> +}
> +
>  /* caller must hold sessionid_lock */
>  static struct nfsd4_session *
>  find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid)
> @@ -1186,7 +1321,67 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  		     struct nfsd4_compound_state *cstate,
>  		     struct nfsd4_create_session *cr_ses)
>  {
> -	return -1;	/* stub */
> +	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
> +	struct nfs4_client *conf, *unconf;
> +	int status = 0;
> +
> +	nfs4_lock_state();
> +	unconf = find_unconfirmed_client(&cr_ses->clientid);
> +	conf = find_confirmed_client(&cr_ses->clientid);
> +
> +	if (conf) {
> +		status = nfs_ok;
> +		if (conf->cl_seqid == cr_ses->seqid) {
> +			dprintk("Got a create_session replay! seqid= %d\n",
> +				conf->cl_seqid);
> +			goto out_replay;
> +		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
> +			status = nfserr_seq_misordered;
> +			dprintk("Sequence misordered!\n");
> +			dprintk("Expected seqid= %d but got seqid= %d\n",
> +				conf->cl_seqid, cr_ses->seqid);
> +			goto out;
> +		}
> +		conf->cl_seqid++;
> +	} else if (unconf) {
> +		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
> +		    (ip_addr != unconf->cl_addr)) {

Again, I don't believe the nfsv4/4.1 specs require the client to keep to
a single ip address, so we shouldn't be doing ip address checks.

--b.

> +			status = nfserr_clid_inuse;
> +			goto out;
> +		}
> +
> +		if (unconf->cl_seqid != cr_ses->seqid) {
> +			status = nfserr_seq_misordered;
> +			goto out;
> +		}
> +
> +		move_to_confirmed(unconf);
> +
> +		/*
> +		 * We do not support RDMA or persistent sessions
> +		 */
> +		cr_ses->flags &= ~SESSION4_PERSIST;
> +		cr_ses->flags &= ~SESSION4_RDMA;
> +
> +		conf = unconf;
> +	} else {
> +		status = nfserr_stale_clientid;
> +		goto out;

Is this the right error?

--b.

> +	}
> +
> +	status = alloc_init_session(rqstp, conf, cr_ses);
> +	if (status)
> +		goto out;
> +
> +out_replay:
> +	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
> +	       NFS4_MAX_SESSIONID_LEN);
> +	cr_ses->seqid = conf->cl_seqid;
> +
> +out:
> +	nfs4_unlock_state();
> +	dprintk("%s returns %d\n", __func__, ntohl(status));
> +	return status;
>  }
>  
>  __be32
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index 57afb33..60db854 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -1100,7 +1100,108 @@ static __be32
>  nfsd4_decode_create_session(struct nfsd4_compoundargs *argp,
>  			    struct nfsd4_create_session *sess)
>  {
> -	return nfserr_opnotsupp;	/* stub */
> +	DECODE_HEAD;
> +
> +	u32 dummy;
> +	char *machine_name;
> +	int i;
> +	int nr_secflavs;
> +
> +	READ_BUF(16);
> +	COPYMEM(&sess->clientid, 8);
> +	READ32(sess->seqid);
> +	READ32(sess->flags);
> +
> +	/* Fore channel attrs */
> +	READ_BUF(28);
> +	READ32(dummy); /* headerpadsz is always 0 */
> +	READ32(sess->fore_channel.maxreq_sz);
> +	READ32(sess->fore_channel.maxresp_sz);
> +	READ32(sess->fore_channel.maxresp_cached);
> +	READ32(sess->fore_channel.maxops);
> +	READ32(sess->fore_channel.maxreqs);
> +	READ32(sess->fore_channel.nr_rdma_attrs);
> +	if (sess->fore_channel.nr_rdma_attrs == 1) {
> +		READ_BUF(4);
> +		READ32(sess->fore_channel.rdma_attrs);
> +	} else if (sess->fore_channel.nr_rdma_attrs > 1) {
> +		dprintk("Too many fore channel attr bitmaps!\n");
> +		goto xdr_error;
> +	}
> +
> +	/* Back channel attrs */
> +	READ_BUF(28);
> +	READ32(dummy); /* headerpadsz is always 0 */
> +	READ32(sess->back_channel.maxreq_sz);
> +	READ32(sess->back_channel.maxresp_sz);
> +	READ32(sess->back_channel.maxresp_cached);
> +	READ32(sess->back_channel.maxops);
> +	READ32(sess->back_channel.maxreqs);
> +	READ32(sess->back_channel.nr_rdma_attrs);
> +	if (sess->back_channel.nr_rdma_attrs == 1) {
> +		READ_BUF(4);
> +		READ32(sess->back_channel.rdma_attrs);
> +	} else if (sess->back_channel.nr_rdma_attrs > 1) {
> +		dprintk("Too many back channel attr bitmaps!\n");
> +		goto xdr_error;
> +	}
> +
> +	READ_BUF(8);
> +	READ32(sess->callback_prog);
> +
> +	/* callback_sec_params4 */
> +	READ32(nr_secflavs);
> +	for (i = 0; i < nr_secflavs; ++i) {
> +		READ_BUF(4);
> +		READ32(dummy);
> +		switch (dummy) {
> +		case RPC_AUTH_NULL:
> +			/* Nothing to read */
> +			break;
> +		case RPC_AUTH_UNIX:
> +			READ_BUF(8);
> +			/* stamp */
> +			READ32(dummy);
> +
> +			/* machine name */
> +			READ32(dummy);
> +			READ_BUF(dummy);
> +			SAVEMEM(machine_name, dummy);
> +
> +			/* uid, gid */
> +			READ_BUF(8);
> +			READ32(sess->uid);
> +			READ32(sess->gid);
> +
> +			/* more gids */
> +			READ_BUF(4);
> +			READ32(dummy);
> +			READ_BUF(dummy * 4);
> +			for (i = 0; i < dummy; ++i)
> +				READ32(dummy);
> +			break;
> +		case RPC_AUTH_GSS:
> +			dprintk("RPC_AUTH_GSS callback secflavor "
> +				"not supported!\n");
> +			READ_BUF(8);
> +			/* gcbp_service */
> +			READ32(dummy);
> +			/* gcbp_handle_from_server */
> +			READ32(dummy);
> +			READ_BUF(dummy);
> +			p += XDR_QUADLEN(dummy);
> +			/* gcbp_handle_from_client */
> +			READ_BUF(4);
> +			READ32(dummy);
> +			READ_BUF(dummy);
> +			p += XDR_QUADLEN(dummy);
> +			break;
> +		default:
> +			dprintk("Illegal callback secflavor\n");
> +			return nfserr_inval;
> +		}
> +	}
> +	DECODE_TAIL;
>  }
>  
>  static __be32
> @@ -2829,7 +2930,49 @@ static __be32
>  nfsd4_encode_create_session(struct nfsd4_compoundres *resp, int nfserr,
>  			    struct nfsd4_create_session *sess)
>  {
> -	/* stub */
> +	ENCODE_HEAD;
> +
> +	if (nfserr)
> +		goto out;
> +
> +	RESERVE_SPACE(24);
> +	WRITEMEM(sess->sessionid.data, NFS4_MAX_SESSIONID_LEN);
> +	WRITE32(sess->seqid);
> +	WRITE32(sess->flags);
> +	ADJUST_ARGS();
> +
> +	RESERVE_SPACE(28);
> +	WRITE32(0); /* headerpadsz */
> +	WRITE32(sess->fore_channel.maxreq_sz);
> +	WRITE32(sess->fore_channel.maxresp_sz);
> +	WRITE32(sess->fore_channel.maxresp_cached);
> +	WRITE32(sess->fore_channel.maxops);
> +	WRITE32(sess->fore_channel.maxreqs);
> +	WRITE32(sess->fore_channel.nr_rdma_attrs);
> +	ADJUST_ARGS();
> +
> +	if (sess->fore_channel.nr_rdma_attrs) {
> +		RESERVE_SPACE(4);
> +		WRITE32(sess->fore_channel.rdma_attrs);
> +		ADJUST_ARGS();
> +	}
> +
> +	RESERVE_SPACE(28);
> +	WRITE32(0); /* headerpadsz */
> +	WRITE32(sess->back_channel.maxreq_sz);
> +	WRITE32(sess->back_channel.maxresp_sz);
> +	WRITE32(sess->back_channel.maxresp_cached);
> +	WRITE32(sess->back_channel.maxops);
> +	WRITE32(sess->back_channel.maxreqs);
> +	WRITE32(sess->back_channel.nr_rdma_attrs);
> +	ADJUST_ARGS();
> +
> +	if (sess->back_channel.nr_rdma_attrs) {
> +		RESERVE_SPACE(4);
> +		WRITE32(sess->back_channel.rdma_attrs);
> +		ADJUST_ARGS();
> +	}
> +out:
>  	return nfserr;
>  }
>  
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 8ca6a82..98d7b1c 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -99,8 +99,12 @@ struct nfs4_callback {
>  	struct rpc_clnt *       cb_client;
>  };
>  
> +/* Maximum number of slots per session. 128 is useful for long haul TCP */
> +#define NFSD_MAX_SLOTS_PER_SESSION	128
>  /* Maximum number of pages per slot cache entry */
>  #define NFSD_PAGES_PER_SLOT	1
> +/* Maximum number of operations per session compound */
> +#define NFSD_MAX_OPS_PER_COMPOUND	16
>  
>  struct nfsd4_cache_entry {
>  	__be32		ce_status;
> @@ -188,6 +192,9 @@ struct nfs4_client {
>  	struct list_head	cl_sessions;
>  	u32			cl_seqid;       /* seqid for create_session */
>  	u32			cl_exchange_flags;
> +	struct nfs4_sessionid	cl_sessionid;
> +
> +	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
>  #endif /* CONFIG_NFSD_V4_1 */
>  };
>  
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 5c0d376..c7bf0a1 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -360,8 +360,27 @@ struct nfsd4_exchange_id {
>  	int		spa_how;
>  };
>  
> +struct nfsd4_channel_attrs {
> +	u32		headerpadsz;
> +	u32		maxreq_sz;
> +	u32		maxresp_sz;
> +	u32		maxresp_cached;
> +	u32		maxops;
> +	u32		maxreqs;
> +	u32		nr_rdma_attrs;
> +	u32		rdma_attrs;
> +};
> +
>  struct nfsd4_create_session {
> -	int	foo;	/* stub */
> +	clientid_t		clientid;
> +	struct nfs4_sessionid	sessionid;
> +	u32			seqid;
> +	u32			flags;
> +	struct nfsd4_channel_attrs fore_channel;
> +	struct nfsd4_channel_attrs back_channel;
> +	u32			callback_prog;
> +	u32			uid;
> +	u32			gid;
>  };
>  
>  struct nfsd4_sequence {
> -- 
> 1.6.2.1
> 

[PATCH v2 24/47] nfsd41: Add a create session replay cache

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Replace the nfs4_client cl_seqid field with a single struct nfs41_slot used
for the create session replay cache.

The CREATE_SESSION slot sets the sl_session pointer to NULL. Otherwise, the
slot and it's replay cache are used just like the session slots.

Fix unconfirmed create_session replay response by initializing the
create_session slot sequence id to 0.

A future patch will set the CREATE_SESSION cache when a SEQUENCE operation
preceeds the CREATE_SESSION operation. This compound is currently only cached
in the session slot table.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[nfsd41: use bool inuse for slot state]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++-------------
 include/linux/nfsd/state.h |    2 +-
 2 files changed, 34 insertions(+), 15 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index e4e2c19..61af434 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -662,6 +662,10 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+#if defined(CONFIG_NFSD_V4_1)
+	nfsd4_release_respages(clp->cl_slot.sl_cache_entry.ce_respages,
+			     clp->cl_slot.sl_cache_entry.ce_resused);
+#endif /* CONFIG_NFSD_V4_1 */
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1055,12 +1059,14 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
 {
 	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
 	struct svc_rqst *rqstp = resp->rqstp;
+	struct nfsd4_compoundargs *args = rqstp->rq_argp;
+	struct nfsd4_op *op = &args->ops[resp->opcnt];
 	struct kvec *resv = &rqstp->rq_res.head[0];
 
 	dprintk("--> %s entry %p\n", __func__, entry);
 
-	/* Don't cache a failed OP_SEQUENCE */
-	if (resp->opcnt == 1 && resp->cstate.status)
+	/* Don't cache a failed OP_SEQUENCE. */
+	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
 		return;
 	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
 	entry->ce_resused = rqstp->rq_resused;
@@ -1276,11 +1282,12 @@ out_copy:
 	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
 	exid->clientid.cl_id = new->cl_clientid.cl_id;
 
-	new->cl_seqid = exid->seqid = 1;
+	new->cl_slot.sl_seqid = 0;
+	exid->seqid = 1;
 	nfsd4_set_ex_flags(new, exid);
 
 	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
-		new->cl_seqid, new->cl_exchange_flags);
+		new->cl_slot.sl_seqid, new->cl_exchange_flags);
 	status = nfs_ok;
 
 out:
@@ -1322,7 +1329,9 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		     struct nfsd4_create_session *cr_ses)
 {
 	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfs4_client *conf, *unconf;
+	struct nfsd4_slot *slot = NULL;
 	int status = 0;
 
 	nfs4_lock_state();
@@ -1330,19 +1339,24 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 	conf = find_confirmed_client(&cr_ses->clientid);
 
 	if (conf) {
-		status = nfs_ok;
-		if (conf->cl_seqid == cr_ses->seqid) {
+		slot = &conf->cl_slot;
+		status = check_slot_seqid(cr_ses->seqid, slot);
+		if (status == nfserr_replay_cache) {
 			dprintk("Got a create_session replay! seqid= %d\n",
-				conf->cl_seqid);
-			goto out_replay;
-		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
+				slot->sl_seqid);
+			cstate->slot = slot;
+			cstate->status = status;
+			/* Return the cached reply status */
+			status = nfsd4_replay_cache_entry(resp);
+			goto out;
+		} else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
 			status = nfserr_seq_misordered;
 			dprintk("Sequence misordered!\n");
 			dprintk("Expected seqid= %d but got seqid= %d\n",
-				conf->cl_seqid, cr_ses->seqid);
+				slot->sl_seqid, cr_ses->seqid);
 			goto out;
 		}
-		conf->cl_seqid++;
+		conf->cl_slot.sl_seqid++;
 	} else if (unconf) {
 		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
 		    (ip_addr != unconf->cl_addr)) {
@@ -1350,11 +1364,15 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 			goto out;
 		}
 
-		if (unconf->cl_seqid != cr_ses->seqid) {
+		slot = &unconf->cl_slot;
+		status = check_slot_seqid(cr_ses->seqid, slot);
+		if (status) {
+			/* an unconfirmed replay returns misordered */
 			status = nfserr_seq_misordered;
 			goto out;
 		}
 
+		slot->sl_seqid++; /* from 0 to 1 */
 		move_to_confirmed(unconf);
 
 		/*
@@ -1373,11 +1391,12 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 	if (status)
 		goto out;
 
-out_replay:
 	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
 	       NFS4_MAX_SESSIONID_LEN);
-	cr_ses->seqid = conf->cl_seqid;
+	cr_ses->seqid = slot->sl_seqid;
 
+	slot->sl_inuse = true;
+	cstate->slot = slot;
 out:
 	nfs4_unlock_state();
 	dprintk("%s returns %d\n", __func__, ntohl(status));
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 98d7b1c..49d89fd 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -190,7 +190,7 @@ struct nfs4_client {
 	u32			cl_firststate;	/* recovery dir creation */
 #ifdef CONFIG_NFSD_V4_1
 	struct list_head	cl_sessions;
-	u32			cl_seqid;       /* seqid for create_session */
+	struct nfsd4_slot	cl_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
 
-- 
1.6.2.1

Re: [PATCH v2 24/47] nfsd41: Add a create session replay cache

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:33:02AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Replace the nfs4_client cl_seqid field with a single struct nfs41_slot used
> for the create session replay cache.
> 
> The CREATE_SESSION slot sets the sl_session pointer to NULL. Otherwise, the
> slot and it's replay cache are used just like the session slots.
> 
> Fix unconfirmed create_session replay response by initializing the
> create_session slot sequence id to 0.
> 
> A future patch will set the CREATE_SESSION cache when a SEQUENCE operation
> preceeds the CREATE_SESSION operation. This compound is currently only cached
> in the session slot table.
> 
> Signed-off-by: Andy Adamson<andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [nfsd41: use bool inuse for slot state]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++-------------
>  include/linux/nfsd/state.h |    2 +-
>  2 files changed, 34 insertions(+), 15 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index e4e2c19..61af434 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -662,6 +662,10 @@ static inline void
>  free_client(struct nfs4_client *clp)
>  {
>  	shutdown_callback_client(clp);
> +#if defined(CONFIG_NFSD_V4_1)
> +	nfsd4_release_respages(clp->cl_slot.sl_cache_entry.ce_respages,
> +			     clp->cl_slot.sl_cache_entry.ce_resused);
> +#endif /* CONFIG_NFSD_V4_1 */

Hide these ifdef's somehow.  And maybe make a free_slot that does this
and that's called from here and free_session.

>  	if (clp->cl_cred.cr_group_info)
>  		put_group_info(clp->cl_cred.cr_group_info);
>  	kfree(clp->cl_principal);
> @@ -1055,12 +1059,14 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
>  {
>  	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
>  	struct svc_rqst *rqstp = resp->rqstp;
> +	struct nfsd4_compoundargs *args = rqstp->rq_argp;
> +	struct nfsd4_op *op = &args->ops[resp->opcnt];
>  	struct kvec *resv = &rqstp->rq_res.head[0];
>  
>  	dprintk("--> %s entry %p\n", __func__, entry);
>  
> -	/* Don't cache a failed OP_SEQUENCE */
> -	if (resp->opcnt == 1 && resp->cstate.status)
> +	/* Don't cache a failed OP_SEQUENCE. */
> +	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)

Why is this in this patch?

--b.

>  		return;
>  	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
>  	entry->ce_resused = rqstp->rq_resused;
> @@ -1276,11 +1282,12 @@ out_copy:
>  	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
>  	exid->clientid.cl_id = new->cl_clientid.cl_id;
>  
> -	new->cl_seqid = exid->seqid = 1;
> +	new->cl_slot.sl_seqid = 0;
> +	exid->seqid = 1;
>  	nfsd4_set_ex_flags(new, exid);
>  
>  	dprintk("nfsd4_exchange_id seqid %d flags %x\n",
> -		new->cl_seqid, new->cl_exchange_flags);
> +		new->cl_slot.sl_seqid, new->cl_exchange_flags);
>  	status = nfs_ok;
>  
>  out:
> @@ -1322,7 +1329,9 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  		     struct nfsd4_create_session *cr_ses)
>  {
>  	u32 ip_addr = svc_addr_in(rqstp)->sin_addr.s_addr;
> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>  	struct nfs4_client *conf, *unconf;
> +	struct nfsd4_slot *slot = NULL;
>  	int status = 0;
>  
>  	nfs4_lock_state();
> @@ -1330,19 +1339,24 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  	conf = find_confirmed_client(&cr_ses->clientid);
>  
>  	if (conf) {
> -		status = nfs_ok;
> -		if (conf->cl_seqid == cr_ses->seqid) {
> +		slot = &conf->cl_slot;
> +		status = check_slot_seqid(cr_ses->seqid, slot);
> +		if (status == nfserr_replay_cache) {
>  			dprintk("Got a create_session replay! seqid= %d\n",
> -				conf->cl_seqid);
> -			goto out_replay;
> -		} else if (cr_ses->seqid != conf->cl_seqid + 1) {
> +				slot->sl_seqid);
> +			cstate->slot = slot;
> +			cstate->status = status;
> +			/* Return the cached reply status */
> +			status = nfsd4_replay_cache_entry(resp);
> +			goto out;
> +		} else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
>  			status = nfserr_seq_misordered;
>  			dprintk("Sequence misordered!\n");
>  			dprintk("Expected seqid= %d but got seqid= %d\n",
> -				conf->cl_seqid, cr_ses->seqid);
> +				slot->sl_seqid, cr_ses->seqid);
>  			goto out;
>  		}
> -		conf->cl_seqid++;
> +		conf->cl_slot.sl_seqid++;
>  	} else if (unconf) {
>  		if (!same_creds(&unconf->cl_cred, &rqstp->rq_cred) ||
>  		    (ip_addr != unconf->cl_addr)) {
> @@ -1350,11 +1364,15 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  			goto out;
>  		}
>  
> -		if (unconf->cl_seqid != cr_ses->seqid) {
> +		slot = &unconf->cl_slot;
> +		status = check_slot_seqid(cr_ses->seqid, slot);
> +		if (status) {
> +			/* an unconfirmed replay returns misordered */
>  			status = nfserr_seq_misordered;
>  			goto out;
>  		}
>  
> +		slot->sl_seqid++; /* from 0 to 1 */
>  		move_to_confirmed(unconf);
>  
>  		/*
> @@ -1373,11 +1391,12 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  	if (status)
>  		goto out;
>  
> -out_replay:
>  	memcpy(cr_ses->sessionid.data, conf->cl_sessionid.data,
>  	       NFS4_MAX_SESSIONID_LEN);
> -	cr_ses->seqid = conf->cl_seqid;
> +	cr_ses->seqid = slot->sl_seqid;
>  
> +	slot->sl_inuse = true;
> +	cstate->slot = slot;
>  out:
>  	nfs4_unlock_state();
>  	dprintk("%s returns %d\n", __func__, ntohl(status));
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 98d7b1c..49d89fd 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -190,7 +190,7 @@ struct nfs4_client {
>  	u32			cl_firststate;	/* recovery dir creation */
>  #ifdef CONFIG_NFSD_V4_1
>  	struct list_head	cl_sessions;
> -	u32			cl_seqid;       /* seqid for create_session */
> +	struct nfsd4_slot	cl_slot;	/* create_session slot */
>  	u32			cl_exchange_flags;
>  	struct nfs4_sessionid	cl_sessionid;
>  
> -- 
> 1.6.2.1
> 

[PATCH v2 25/47] nfsd41: non-page DRC for solo sequence responses

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

A session inactivity time compound (lease renewal) or a compound where the
sequence operation has sa_cachethis set to FALSE do not require any pages
to be held in the v4.1 DRC. This is because struct nfsd4_slot is already
caching the session information.

Add logic to the nfs41 server to not cache response pages for solo sequence
responses.

Return nfserr_replay_uncached_rep on the operation following the sequence
operation when sa_cachethis is FALSE.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c         |   34 +++++++++++++++++++++++++++++-
 fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++++++++++-----
 fs/nfsd/nfs4xdr.c          |    5 ++-
 include/linux/nfsd/state.h |    1 +
 include/linux/nfsd/xdr4.h  |   15 +++++++++++++-
 5 files changed, 91 insertions(+), 11 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index bdbeb87..a273023 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -828,6 +828,34 @@ static struct nfsd4_operation nfsd4_ops[];
 static const char *nfsd4_op_name(unsigned opnum);
 
 /*
+ * This is a replay of a compound for which no cache entry pages
+ * were used. Encode the sequence operation, and if cachethis is FALSE
+ * encode the uncache rep error on the next operation.
+ */
+static __be32
+nfsd4_enc_no_page_replay(struct nfsd4_compoundargs *args,
+			 struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_op *op;
+
+	dprintk("--> %s resp->opcnt %d ce_cachethis %u \n", __func__,
+		resp->opcnt, resp->cstate.slot->sl_cache_entry.ce_cachethis);
+
+	/* Encode the replayed sequence operation */
+	BUG_ON(resp->opcnt != 1);
+	op = &args->ops[resp->opcnt - 1];
+	nfsd4_encode_operation(resp, op);
+
+	/*return nfserr_retry_uncached_rep in next operation. */
+	if (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0) {
+		op = &args->ops[resp->opcnt++];
+		op->status = nfserr_retry_uncached_rep;
+		nfsd4_encode_operation(resp, op);
+	}
+	return op->status;
+}
+
+/*
  * COMPOUND call.
  */
 static __be32
@@ -879,7 +907,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 		dprintk("nfsv4 compound op #%d/%d: %d (%s)\n",
 			resp->opcnt, args->opcnt, op->opnum,
 			nfsd4_op_name(op->opnum));
-
 		/*
 		 * The XDR decode routines may have pre-set op->status;
 		 * for example, if there is a miscellaneous XDR error
@@ -923,7 +950,10 @@ encode_op:
 		/* Only from SEQUENCE or CREATE_SESSION */
 		if (resp->cstate.status == nfserr_replay_cache) {
 			dprintk("%s NFS4.1 replay from cache\n", __func__);
-			status = op->status;
+			if (nfsd4_no_page_in_cache(resp))
+				status = nfsd4_enc_no_page_replay(args, resp);
+			else
+				status = op->status;
 			goto out;
 		}
 		if (op->status == nfserr_replay_me) {
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 61af434..f42cda9 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1068,17 +1068,31 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
 	/* Don't cache a failed OP_SEQUENCE. */
 	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
 		return;
+
 	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
+	entry->ce_opcnt = resp->opcnt;
+	entry->ce_status = resp->cstate.status;
+
+	/*
+	 * Don't need a page to cache just the sequence operation - the slot
+	 * does this for us!
+	 */
+
+	if (nfsd4_no_page_in_cache(resp)) {
+		entry->ce_resused = 0;
+		entry->ce_rpchdrlen = 0;
+		dprintk("%s Just cache SEQUENCE. ce_cachethis %d\n", __func__,
+			resp->cstate.slot->sl_cache_entry.ce_cachethis);
+		return;
+	}
 	entry->ce_resused = rqstp->rq_resused;
 	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
 		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
 	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
 			 entry->ce_resused);
-	entry->ce_status = resp->cstate.status;
 	entry->ce_datav.iov_base = resp->cstate.statp;
 	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
 				(char *)page_address(rqstp->rq_respages[0]));
-	entry->ce_opcnt = resp->opcnt;
 	/* Current request rpc header length*/
 	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
 				(char *)page_address(rqstp->rq_respages[0]);
@@ -1117,13 +1131,28 @@ nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
  * cached page.  Replace any futher replay pages from the cache.
  */
 __be32
-nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
+nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
+			 struct nfsd4_sequence *seq)
 {
 	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
 	__be32 status;
 
 	dprintk("--> %s entry %p\n", __func__, entry);
 
+	/*
+	 * If this is just the sequence operation, we did not keep
+	 * a page in the cache entry because we can just use the
+	 * slot info stored in struct nfsd4_sequence that was checked
+	 * against the slot in nfsd4_sequence().
+	 *
+	 * This occurs when seq->cachethis is FALSE, or when the client
+	 * session inactivity timer fires and a solo sequence operation
+	 * is sent (lease renewal).
+	 */
+	if (seq && nfsd4_no_page_in_cache(resp)) {
+		seq->maxslots = resp->cstate.slot->sl_session->se_fnumslots;
+		return nfs_ok;
+	}
 
 	if (!nfsd41_copy_replay_data(resp, entry)) {
 		/*
@@ -1347,7 +1376,7 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 			cstate->slot = slot;
 			cstate->status = status;
 			/* Return the cached reply status */
-			status = nfsd4_replay_cache_entry(resp);
+			status = nfsd4_replay_cache_entry(resp, NULL);
 			goto out;
 		} else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
 			status = nfserr_seq_misordered;
@@ -1397,6 +1426,8 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 
 	slot->sl_inuse = true;
 	cstate->slot = slot;
+	/* Ensure a page is used for the cache */
+	slot->sl_cache_entry.ce_cachethis = 1;
 out:
 	nfs4_unlock_state();
 	dprintk("%s returns %d\n", __func__, ntohl(status));
@@ -1441,8 +1472,8 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	if (status == nfserr_replay_cache) {
 		cstate->slot = slot;
 		/* Return the cached reply status and set cstate->status
-		 * for nfsd4_svc_encode_compoundres processing*/
-		status = nfsd4_replay_cache_entry(resp);
+		 * for nfsd4_svc_encode_compoundres processing */
+		status = nfsd4_replay_cache_entry(resp, seq);
 		cstate->status = nfserr_replay_cache;
 		goto replay_cache;
 	}
@@ -1452,6 +1483,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	/* Success! bump slot seqid */
 	slot->sl_inuse = true;
 	slot->sl_seqid = seq->seqid;
+	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
+	/* Always set the cache entry cachethis for solo sequence */
+	if (nfsd4_is_solo_sequence(resp))
+		slot->sl_cache_entry.ce_cachethis = 1;
 
 	cstate->slot = slot;
 
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 60db854..a8bb04a 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -2984,7 +2984,7 @@ nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
 	return nfserr;
 }
 
-static __be32
+__be32
 nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
 		      struct nfsd4_sequence *seq)
 {
@@ -3204,7 +3204,8 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 	BUG_ON(iov->iov_len > PAGE_SIZE);
 #ifdef CONFIG_NFSD_V4_1
 	if (resp->cstate.slot != NULL) {
-		if (resp->cstate.status == nfserr_replay_cache) {
+		if (resp->cstate.status == nfserr_replay_cache &&
+				!nfsd4_no_page_in_cache(resp)) {
 			iov->iov_len = resp->cstate.iovlen;
 		} else {
 			nfsd4_set_cache_entry(resp);
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 49d89fd..47c7836 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -110,6 +110,7 @@ struct nfsd4_cache_entry {
 	__be32		ce_status;
 	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
 	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
+	int		ce_cachethis;
 	short		ce_resused;
 	int		ce_opcnt;
 	int		ce_rpchdrlen;
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index c7bf0a1..641e5cd 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -482,6 +482,18 @@ struct nfsd4_compoundres {
 	struct nfsd4_compound_state	cstate;
 };
 
+static inline u32 nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
+{
+	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
+	return args->opcnt == 1 ? 1 : 0;
+}
+
+static inline u32 nfsd4_no_page_in_cache(struct nfsd4_compoundres *resp)
+{
+	return (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0 ||
+			nfsd4_is_solo_sequence(resp));
+}
+
 #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
 
 static inline void
@@ -513,7 +525,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
 		struct nfsd4_setclientid_confirm *setclientid_confirm);
 #if defined(CONFIG_NFSD_V4_1)
 extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
-extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
+extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
+		struct nfsd4_sequence *seq);
 extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *,
 struct nfsd4_exchange_id *);
-- 
1.6.2.1

Re: [PATCH v2 25/47] nfsd41: non-page DRC for solo sequence responses

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:33:11AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> A session inactivity time compound (lease renewal) or a compound where the
> sequence operation has sa_cachethis set to FALSE do not require any pages
> to be held in the v4.1 DRC. This is because struct nfsd4_slot is already
> caching the session information.
> 
> Add logic to the nfs41 server to not cache response pages for solo sequence
> responses.
> 
> Return nfserr_replay_uncached_rep on the operation following the sequence
> operation when sa_cachethis is FALSE.
> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4proc.c         |   34 +++++++++++++++++++++++++++++-
>  fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++++++++++-----
>  fs/nfsd/nfs4xdr.c          |    5 ++-
>  include/linux/nfsd/state.h |    1 +
>  include/linux/nfsd/xdr4.h  |   15 +++++++++++++-
>  5 files changed, 91 insertions(+), 11 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index bdbeb87..a273023 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -828,6 +828,34 @@ static struct nfsd4_operation nfsd4_ops[];
>  static const char *nfsd4_op_name(unsigned opnum);
>  
>  /*
> + * This is a replay of a compound for which no cache entry pages
> + * were used. Encode the sequence operation, and if cachethis is FALSE
> + * encode the uncache rep error on the next operation.
> + */
> +static __be32
> +nfsd4_enc_no_page_replay(struct nfsd4_compoundargs *args,
> +			 struct nfsd4_compoundres *resp)
> +{
> +	struct nfsd4_op *op;
> +
> +	dprintk("--> %s resp->opcnt %d ce_cachethis %u \n", __func__,
> +		resp->opcnt, resp->cstate.slot->sl_cache_entry.ce_cachethis);
> +
> +	/* Encode the replayed sequence operation */
> +	BUG_ON(resp->opcnt != 1);
> +	op = &args->ops[resp->opcnt - 1];
> +	nfsd4_encode_operation(resp, op);
> +
> +	/*return nfserr_retry_uncached_rep in next operation. */
> +	if (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0) {
> +		op = &args->ops[resp->opcnt++];
> +		op->status = nfserr_retry_uncached_rep;
> +		nfsd4_encode_operation(resp, op);

Encoding both operations here makes me very nervous, but I haven't
thought it through.

> +	}
> +	return op->status;
> +}
> +
> +/*
>   * COMPOUND call.
>   */
>  static __be32
> @@ -879,7 +907,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>  		dprintk("nfsv4 compound op #%d/%d: %d (%s)\n",
>  			resp->opcnt, args->opcnt, op->opnum,
>  			nfsd4_op_name(op->opnum));
> -
>  		/*
>  		 * The XDR decode routines may have pre-set op->status;
>  		 * for example, if there is a miscellaneous XDR error
> @@ -923,7 +950,10 @@ encode_op:
>  		/* Only from SEQUENCE or CREATE_SESSION */
>  		if (resp->cstate.status == nfserr_replay_cache) {
>  			dprintk("%s NFS4.1 replay from cache\n", __func__);
> -			status = op->status;
> +			if (nfsd4_no_page_in_cache(resp))

Why not just call that nfsd4_not_cached()?

> +				status = nfsd4_enc_no_page_replay(args, resp);

and nfsd4_enc_uncached_replay()?  (The "no_page" this is a technical
detail of the current caching implementation.)

> +			else
> +				status = op->status;
>  			goto out;
>  		}
>  		if (op->status == nfserr_replay_me) {
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 61af434..f42cda9 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -1068,17 +1068,31 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
>  	/* Don't cache a failed OP_SEQUENCE. */
>  	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
>  		return;
> +
>  	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
> +	entry->ce_opcnt = resp->opcnt;
> +	entry->ce_status = resp->cstate.status;
> +
> +	/*
> +	 * Don't need a page to cache just the sequence operation - the slot
> +	 * does this for us!
> +	 */
> +
> +	if (nfsd4_no_page_in_cache(resp)) {
> +		entry->ce_resused = 0;
> +		entry->ce_rpchdrlen = 0;
> +		dprintk("%s Just cache SEQUENCE. ce_cachethis %d\n", __func__,
> +			resp->cstate.slot->sl_cache_entry.ce_cachethis);
> +		return;
> +	}

Do we *ever* actually need to cache the initial sequence op?  Should we
only be storing subsequent ops in the reply cache?

>  	entry->ce_resused = rqstp->rq_resused;
>  	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
>  		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
>  	nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
>  			 entry->ce_resused);
> -	entry->ce_status = resp->cstate.status;
>  	entry->ce_datav.iov_base = resp->cstate.statp;
>  	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
>  				(char *)page_address(rqstp->rq_respages[0]));
> -	entry->ce_opcnt = resp->opcnt;
>  	/* Current request rpc header length*/
>  	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
>  				(char *)page_address(rqstp->rq_respages[0]);
> @@ -1117,13 +1131,28 @@ nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
>   * cached page.  Replace any futher replay pages from the cache.
>   */
>  __be32
> -nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
> +nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
> +			 struct nfsd4_sequence *seq)
>  {
>  	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
>  	__be32 status;
>  
>  	dprintk("--> %s entry %p\n", __func__, entry);
>  
> +	/*
> +	 * If this is just the sequence operation, we did not keep
> +	 * a page in the cache entry because we can just use the
> +	 * slot info stored in struct nfsd4_sequence that was checked
> +	 * against the slot in nfsd4_sequence().
> +	 *
> +	 * This occurs when seq->cachethis is FALSE, or when the client
> +	 * session inactivity timer fires and a solo sequence operation
> +	 * is sent (lease renewal).
> +	 */
> +	if (seq && nfsd4_no_page_in_cache(resp)) {
> +		seq->maxslots = resp->cstate.slot->sl_session->se_fnumslots;
> +		return nfs_ok;
> +	}
>  
>  	if (!nfsd41_copy_replay_data(resp, entry)) {
>  		/*
> @@ -1347,7 +1376,7 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  			cstate->slot = slot;
>  			cstate->status = status;
>  			/* Return the cached reply status */
> -			status = nfsd4_replay_cache_entry(resp);
> +			status = nfsd4_replay_cache_entry(resp, NULL);
>  			goto out;
>  		} else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
>  			status = nfserr_seq_misordered;
> @@ -1397,6 +1426,8 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>  
>  	slot->sl_inuse = true;
>  	cstate->slot = slot;
> +	/* Ensure a page is used for the cache */
> +	slot->sl_cache_entry.ce_cachethis = 1;
>  out:
>  	nfs4_unlock_state();
>  	dprintk("%s returns %d\n", __func__, ntohl(status));
> @@ -1441,8 +1472,8 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>  	if (status == nfserr_replay_cache) {
>  		cstate->slot = slot;
>  		/* Return the cached reply status and set cstate->status
> -		 * for nfsd4_svc_encode_compoundres processing*/
> -		status = nfsd4_replay_cache_entry(resp);
> +		 * for nfsd4_svc_encode_compoundres processing */

The comment typo-fix doesn't belong in this patch.

> +		status = nfsd4_replay_cache_entry(resp, seq);
>  		cstate->status = nfserr_replay_cache;
>  		goto replay_cache;
>  	}
> @@ -1452,6 +1483,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>  	/* Success! bump slot seqid */
>  	slot->sl_inuse = true;
>  	slot->sl_seqid = seq->seqid;
> +	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
> +	/* Always set the cache entry cachethis for solo sequence */
> +	if (nfsd4_is_solo_sequence(resp))
> +		slot->sl_cache_entry.ce_cachethis = 1;
>  
>  	cstate->slot = slot;
>  
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index 60db854..a8bb04a 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -2984,7 +2984,7 @@ nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
>  	return nfserr;
>  }
>  
> -static __be32
> +__be32
>  nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
>  		      struct nfsd4_sequence *seq)
>  {
> @@ -3204,7 +3204,8 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
>  	BUG_ON(iov->iov_len > PAGE_SIZE);
>  #ifdef CONFIG_NFSD_V4_1
>  	if (resp->cstate.slot != NULL) {
> -		if (resp->cstate.status == nfserr_replay_cache) {
> +		if (resp->cstate.status == nfserr_replay_cache &&
> +				!nfsd4_no_page_in_cache(resp)) {
>  			iov->iov_len = resp->cstate.iovlen;
>  		} else {
>  			nfsd4_set_cache_entry(resp);
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 49d89fd..47c7836 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -110,6 +110,7 @@ struct nfsd4_cache_entry {
>  	__be32		ce_status;
>  	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
>  	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
> +	int		ce_cachethis;
>  	short		ce_resused;
>  	int		ce_opcnt;
>  	int		ce_rpchdrlen;
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index c7bf0a1..641e5cd 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -482,6 +482,18 @@ struct nfsd4_compoundres {
>  	struct nfsd4_compound_state	cstate;
>  };
>  
> +static inline u32 nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
> +{
> +	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
> +	return args->opcnt == 1 ? 1 : 0;

Drop the redundant "? 1: 0", and make the return int (or boolean, if you
want).

> +}
> +
> +static inline u32 nfsd4_no_page_in_cache(struct nfsd4_compoundres *resp)

Ditto on the return type.

> +{
> +	return (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0 ||
> +			nfsd4_is_solo_sequence(resp));

Drop the extra parentheses.

--b.

> +}
> +
>  #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
>  
>  static inline void
> @@ -513,7 +525,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>  		struct nfsd4_setclientid_confirm *setclientid_confirm);
>  #if defined(CONFIG_NFSD_V4_1)
>  extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
> -extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
> +extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
> +		struct nfsd4_sequence *seq);
>  extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
>  		struct nfsd4_compound_state *,
>  struct nfsd4_exchange_id *);
> -- 
> 1.6.2.1
> 

Re: [pnfs] [PATCH v2 25/47] nfsd41: non-page DRC for solo sequence responses

[William A. (Andy) Adamson]

On Wed, Apr 1, 2009 at 12:12 AM, J. Bruce Fields <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:33:11AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> A session inactivity time compound (lease renewal) or a compound where the
>> sequence operation has sa_cachethis set to FALSE do not require any pages
>> to be held in the v4.1 DRC. This is because struct nfsd4_slot is already
>> caching the session information.
>>
>> Add logic to the nfs41 server to not cache response pages for solo sequence
>> responses.
>>
>> Return nfserr_replay_uncached_rep on the operation following the sequence
>> operation when sa_cachethis is FALSE.
>>
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4proc.c         |   34 +++++++++++++++++++++++++++++-
>>  fs/nfsd/nfs4state.c        |   47 ++++++++++++++++++++++++++++++++++++++-----
>>  fs/nfsd/nfs4xdr.c          |    5 ++-
>>  include/linux/nfsd/state.h |    1 +
>>  include/linux/nfsd/xdr4.h  |   15 +++++++++++++-
>>  5 files changed, 91 insertions(+), 11 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
>> index bdbeb87..a273023 100644
>> --- a/fs/nfsd/nfs4proc.c
>> +++ b/fs/nfsd/nfs4proc.c
>> @@ -828,6 +828,34 @@ static struct nfsd4_operation nfsd4_ops[];
>>  static const char *nfsd4_op_name(unsigned opnum);
>>
>>  /*
>> + * This is a replay of a compound for which no cache entry pages
>> + * were used. Encode the sequence operation, and if cachethis is FALSE
>> + * encode the uncache rep error on the next operation.
>> + */
>> +static __be32
>> +nfsd4_enc_no_page_replay(struct nfsd4_compoundargs *args,
>> +                      struct nfsd4_compoundres *resp)
>> +{
>> +     struct nfsd4_op *op;
>> +
>> +     dprintk("--> %s resp->opcnt %d ce_cachethis %u \n", __func__,
>> +             resp->opcnt, resp->cstate.slot->sl_cache_entry.ce_cachethis);
>> +
>> +     /* Encode the replayed sequence operation */
>> +     BUG_ON(resp->opcnt != 1);
>> +     op = &args->ops[resp->opcnt - 1];
>> +     nfsd4_encode_operation(resp, op);
>> +
>> +     /*return nfserr_retry_uncached_rep in next operation. */
>> +     if (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0) {
>> +             op = &args->ops[resp->opcnt++];
>> +             op->status = nfserr_retry_uncached_rep;
>> +             nfsd4_encode_operation(resp, op);
>
> Encoding both operations here makes me very nervous, but I haven't
> thought it through.

We only encode the sequence operation here, and set the error in the
next operation which gets encoded in the next nfsd4_proc_compound
loop.

>
>> +     }
>> +     return op->status;
>> +}
>> +
>> +/*
>>   * COMPOUND call.
>>   */
>>  static __be32
>> @@ -879,7 +907,6 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
>>               dprintk("nfsv4 compound op #%d/%d: %d (%s)\n",
>>                       resp->opcnt, args->opcnt, op->opnum,
>>                       nfsd4_op_name(op->opnum));
>> -
>>               /*
>>                * The XDR decode routines may have pre-set op->status;
>>                * for example, if there is a miscellaneous XDR error
>> @@ -923,7 +950,10 @@ encode_op:
>>               /* Only from SEQUENCE or CREATE_SESSION */
>>               if (resp->cstate.status == nfserr_replay_cache) {
>>                       dprintk("%s NFS4.1 replay from cache\n", __func__);
>> -                     status = op->status;
>> +                     if (nfsd4_no_page_in_cache(resp))
>
> Why not just call that nfsd4_not_cached()?


OK

>
>> +                             status = nfsd4_enc_no_page_replay(args, resp);
>
> and nfsd4_enc_uncached_replay()?  (The "no_page" this is a technical
> detail of the current caching implementation.)

OK

>
>> +                     else
>> +                             status = op->status;
>>                       goto out;
>>               }
>>               if (op->status == nfserr_replay_me) {
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index 61af434..f42cda9 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -1068,17 +1068,31 @@ nfsd4_set_cache_entry(struct nfsd4_compoundres *resp)
>>       /* Don't cache a failed OP_SEQUENCE. */
>>       if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
>>               return;
>> +
>>       nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
>> +     entry->ce_opcnt = resp->opcnt;
>> +     entry->ce_status = resp->cstate.status;
>> +
>> +     /*
>> +      * Don't need a page to cache just the sequence operation - the slot
>> +      * does this for us!
>> +      */
>> +
>> +     if (nfsd4_no_page_in_cache(resp)) {
>> +             entry->ce_resused = 0;
>> +             entry->ce_rpchdrlen = 0;
>> +             dprintk("%s Just cache SEQUENCE. ce_cachethis %d\n", __func__,
>> +                     resp->cstate.slot->sl_cache_entry.ce_cachethis);
>> +             return;
>> +     }
>
> Do we *ever* actually need to cache the initial sequence op?  Should we
> only be storing subsequent ops in the reply cache?

We cache encoded pages, or not. If we did not do this, then we would
be mucking with each and every reply just to cache it. e.g. we would
be doing a copy of all the other subsequent ops out of the page into
some other storage. We also need the rpc header of the reply so that
we have the choice later on to compare the original principal with the
incoming replay principal.

To me it makes a lot more sense to do the least amount of work in the
common no-replay case, which is to simply store the page or two of the
reply.

>
>>       entry->ce_resused = rqstp->rq_resused;
>>       if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
>>               entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
>>       nfsd4_move_pages(entry->ce_respages, rqstp->rq_respages,
>>                        entry->ce_resused);
>> -     entry->ce_status = resp->cstate.status;
>>       entry->ce_datav.iov_base = resp->cstate.statp;
>>       entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
>>                               (char *)page_address(rqstp->rq_respages[0]));
>> -     entry->ce_opcnt = resp->opcnt;
>>       /* Current request rpc header length*/
>>       entry->ce_rpchdrlen = (char *)resp->cstate.statp -
>>                               (char *)page_address(rqstp->rq_respages[0]);
>> @@ -1117,13 +1131,28 @@ nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
>>   * cached page.  Replace any futher replay pages from the cache.
>>   */
>>  __be32
>> -nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp)
>> +nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
>> +                      struct nfsd4_sequence *seq)
>>  {
>>       struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
>>       __be32 status;
>>
>>       dprintk("--> %s entry %p\n", __func__, entry);
>>
>> +     /*
>> +      * If this is just the sequence operation, we did not keep
>> +      * a page in the cache entry because we can just use the
>> +      * slot info stored in struct nfsd4_sequence that was checked
>> +      * against the slot in nfsd4_sequence().
>> +      *
>> +      * This occurs when seq->cachethis is FALSE, or when the client
>> +      * session inactivity timer fires and a solo sequence operation
>> +      * is sent (lease renewal).
>> +      */
>> +     if (seq && nfsd4_no_page_in_cache(resp)) {
>> +             seq->maxslots = resp->cstate.slot->sl_session->se_fnumslots;
>> +             return nfs_ok;
>> +     }
>>
>>       if (!nfsd41_copy_replay_data(resp, entry)) {
>>               /*
>> @@ -1347,7 +1376,7 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>>                       cstate->slot = slot;
>>                       cstate->status = status;
>>                       /* Return the cached reply status */
>> -                     status = nfsd4_replay_cache_entry(resp);
>> +                     status = nfsd4_replay_cache_entry(resp, NULL);
>>                       goto out;
>>               } else if (cr_ses->seqid != conf->cl_slot.sl_seqid + 1) {
>>                       status = nfserr_seq_misordered;
>> @@ -1397,6 +1426,8 @@ nfsd4_create_session(struct svc_rqst *rqstp,
>>
>>       slot->sl_inuse = true;
>>       cstate->slot = slot;
>> +     /* Ensure a page is used for the cache */
>> +     slot->sl_cache_entry.ce_cachethis = 1;
>>  out:
>>       nfs4_unlock_state();
>>       dprintk("%s returns %d\n", __func__, ntohl(status));
>> @@ -1441,8 +1472,8 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>>       if (status == nfserr_replay_cache) {
>>               cstate->slot = slot;
>>               /* Return the cached reply status and set cstate->status
>> -              * for nfsd4_svc_encode_compoundres processing*/
>> -             status = nfsd4_replay_cache_entry(resp);
>> +              * for nfsd4_svc_encode_compoundres processing */
>
> The comment typo-fix doesn't belong in this patch.

OK

>
>> +             status = nfsd4_replay_cache_entry(resp, seq);
>>               cstate->status = nfserr_replay_cache;
>>               goto replay_cache;
>>       }
>> @@ -1452,6 +1483,10 @@ nfsd4_sequence(struct svc_rqst *rqstp,
>>       /* Success! bump slot seqid */
>>       slot->sl_inuse = true;
>>       slot->sl_seqid = seq->seqid;
>> +     slot->sl_cache_entry.ce_cachethis = seq->cachethis;
>> +     /* Always set the cache entry cachethis for solo sequence */
>> +     if (nfsd4_is_solo_sequence(resp))
>> +             slot->sl_cache_entry.ce_cachethis = 1;
>>
>>       cstate->slot = slot;
>>
>> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
>> index 60db854..a8bb04a 100644
>> --- a/fs/nfsd/nfs4xdr.c
>> +++ b/fs/nfsd/nfs4xdr.c
>> @@ -2984,7 +2984,7 @@ nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
>>       return nfserr;
>>  }
>>
>> -static __be32
>> +__be32
>>  nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
>>                     struct nfsd4_sequence *seq)
>>  {
>> @@ -3204,7 +3204,8 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
>>       BUG_ON(iov->iov_len > PAGE_SIZE);
>>  #ifdef CONFIG_NFSD_V4_1
>>       if (resp->cstate.slot != NULL) {
>> -             if (resp->cstate.status == nfserr_replay_cache) {
>> +             if (resp->cstate.status == nfserr_replay_cache &&
>> +                             !nfsd4_no_page_in_cache(resp)) {
>>                       iov->iov_len = resp->cstate.iovlen;
>>               } else {
>>                       nfsd4_set_cache_entry(resp);
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index 49d89fd..47c7836 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -110,6 +110,7 @@ struct nfsd4_cache_entry {
>>       __be32          ce_status;
>>       struct kvec     ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
>>       struct page     *ce_respages[NFSD_PAGES_PER_SLOT + 1];
>> +     int             ce_cachethis;
>>       short           ce_resused;
>>       int             ce_opcnt;
>>       int             ce_rpchdrlen;
>> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
>> index c7bf0a1..641e5cd 100644
>> --- a/include/linux/nfsd/xdr4.h
>> +++ b/include/linux/nfsd/xdr4.h
>> @@ -482,6 +482,18 @@ struct nfsd4_compoundres {
>>       struct nfsd4_compound_state     cstate;
>>  };
>>
>> +static inline u32 nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
>> +{
>> +     struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
>> +     return args->opcnt == 1 ? 1 : 0;
>
> Drop the redundant "? 1: 0", and make the return int (or boolean, if you
> want).

OK

>
>> +}
>> +
>> +static inline u32 nfsd4_no_page_in_cache(struct nfsd4_compoundres *resp)
>
> Ditto on the return type.

OK

>
>> +{
>> +     return (resp->cstate.slot->sl_cache_entry.ce_cachethis == 0 ||
>> +                     nfsd4_is_solo_sequence(resp));
>
> Drop the extra parentheses.

OK

>
> --b.
>
>> +}
>> +
>>  #define NFS4_SVC_XDRSIZE             sizeof(struct nfsd4_compoundargs)
>>
>>  static inline void
>> @@ -513,7 +525,8 @@ extern __be32 nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>>               struct nfsd4_setclientid_confirm *setclientid_confirm);
>>  #if defined(CONFIG_NFSD_V4_1)
>>  extern void nfsd4_set_cache_entry(struct nfsd4_compoundres *resp);
>> -extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp);
>> +extern __be32 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
>> +             struct nfsd4_sequence *seq);
>>  extern __be32 nfsd4_exchange_id(struct svc_rqst *rqstp,
>>               struct nfsd4_compound_state *,
>>  struct nfsd4_exchange_id *);
>> --
>> 1.6.2.1
>>
> _______________________________________________
> pNFS mailing list
> pNFS@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
>

[PATCH v2 26/47] nfsd41: destroy_session operation

[Benny Halevy]

Implement the destory_session operation confoming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

[use sessionid_lock spin lock]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c       |   29 ++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c         |    7 +++++--
 include/linux/nfsd/xdr4.h |    2 +-
 3 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f42cda9..9c93f96 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1439,7 +1439,34 @@ nfsd4_destroy_session(struct svc_rqst *r,
 		      struct nfsd4_compound_state *cstate,
 		      struct nfsd4_destroy_session *sessionid)
 {
-	return -1;	/* stub */
+	struct nfsd4_session *ses;
+	u32 status = nfserr_badsession;
+
+	/* Notes:
+	 * - The confirmed nfs4_client->cl_sessionid holds destroyed sessinid
+	 * - Should we return nfserr_back_chan_busy if waiting for
+	 *   callbacks on to-be-destroyed session?
+	 * - Do we need to clear any callback info from previous session?
+	 */
+
+	dump_sessionid(__func__, &sessionid->sessionid);
+	spin_lock(&sessionid_lock);
+	ses = find_in_sessionid_hashtbl(&sessionid->sessionid);
+	if (!ses) {
+		spin_unlock(&sessionid_lock);
+		goto out;
+	}
+
+	unhash_session(ses);
+	spin_unlock(&sessionid_lock);
+
+	/* wait for callbacks */
+	shutdown_callback_client(ses->se_client);
+	nfsd4_put_session(ses);
+	status = nfs_ok;
+out:
+	dprintk("%s returns %d\n", __func__, ntohl(status));
+	return status;
 }
 
 __be32
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a8bb04a..5720aab 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1208,7 +1208,11 @@ static __be32
 nfsd4_decode_destroy_session(struct nfsd4_compoundargs *argp,
 			     struct nfsd4_destroy_session *destroy_session)
 {
-	return nfserr_opnotsupp;	/* stub */
+	DECODE_HEAD;
+	READ_BUF(NFS4_MAX_SESSIONID_LEN);
+	COPYMEM(destroy_session->sessionid.data, NFS4_MAX_SESSIONID_LEN);
+
+	DECODE_TAIL;
 }
 
 static __be32
@@ -2980,7 +2984,6 @@ static __be32
 nfsd4_encode_destroy_session(struct nfsd4_compoundres *resp, int nfserr,
 			     struct nfsd4_destroy_session *destroy_session)
 {
-	/* stub */
 	return nfserr;
 }
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 641e5cd..37a7c51 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -396,7 +396,7 @@ struct nfsd4_sequence {
 };
 
 struct nfsd4_destroy_session {
-	int	foo;	/* stub */
+	struct nfs4_sessionid	sessionid;
 };
 #endif /* CONFIG_NFSD_V4_1 */
 
-- 
1.6.2.1

[PATCH v2 27/47] nfsd41: stateid handling

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

When sessions are used, stateful operation sequenceid and stateid handling
are not used. When sessions are used,  on the first open set the seqid to 1,
mark state confirmed and skip seqid processing.

When sessionas are used the stateid generation number is ignored when it is zero
whereas without sessions bad_stateid or stale stateid is returned.

Add flags to propagate session use to all stateful ops and down to
check_stateid_generation.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
[nfsd4_has_session should return a boolean, not u32]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c         |   17 ++++++++--
 fs/nfsd/nfs4state.c        |   70 ++++++++++++++++++++++++++++++++++---------
 fs/nfsd/nfs4xdr.c          |    2 +-
 include/linux/nfsd/state.h |    1 +
 include/linux/nfsd/xdr4.h  |    8 ++++-
 5 files changed, 77 insertions(+), 21 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index a273023..1d4b2b5 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -179,7 +179,7 @@ nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	nfs4_lock_state();
 
 	/* check seqid for replay. set nfs4_owner */
-	status = nfsd4_process_open1(open);
+	status = nfsd4_process_open1(rqstp, open);
 	if (status == nfserr_replay_me) {
 		struct nfs4_replay *rp = &open->op_stateowner->so_replay;
 		fh_put(&cstate->current_fh);
@@ -504,6 +504,7 @@ nfsd4_read(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	   struct nfsd4_read *read)
 {
 	__be32 status;
+	int flags = RD_STATE;
 
 	/* no need to check permission - this will be done in nfsd_read() */
 
@@ -511,11 +512,13 @@ nfsd4_read(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (read->rd_offset >= OFFSET_MAX)
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	/* check stateid */
 	if ((status = nfs4_preprocess_stateid_op(&cstate->current_fh,
 				&read->rd_stateid,
-				RD_STATE, &read->rd_filp))) {
+				flags, &read->rd_filp))) {
 		dprintk("NFSD: nfsd4_read: couldn't process stateid!\n");
 		goto out;
 	}
@@ -643,11 +646,14 @@ nfsd4_setattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	      struct nfsd4_setattr *setattr)
 {
 	__be32 status = nfs_ok;
+	int flags = WR_STATE;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
 		nfs4_lock_state();
 		status = nfs4_preprocess_stateid_op(&cstate->current_fh,
-			&setattr->sa_stateid, WR_STATE, NULL);
+			&setattr->sa_stateid, flags, NULL);
 		nfs4_unlock_state();
 		if (status) {
 			dprintk("NFSD: nfsd4_setattr: couldn't process stateid!\n");
@@ -679,15 +685,18 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	u32 *p;
 	__be32 status = nfs_ok;
 	unsigned long cnt;
+	int flags = WR_STATE;
 
 	/* no need to check permission - this will be done in nfsd_write() */
 
 	if (write->wr_offset >= OFFSET_MAX)
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	status = nfs4_preprocess_stateid_op(&cstate->current_fh, stateid,
-					WR_STATE, &filp);
+					    flags, &filp);
 	if (filp)
 		get_file(filp);
 	nfs4_unlock_state();
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 9c93f96..bf5b214 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -2199,12 +2199,13 @@ static struct lock_manager_operations nfsd_lease_mng_ops = {
 
 
 __be32
-nfsd4_process_open1(struct nfsd4_open *open)
+nfsd4_process_open1(struct svc_rqst *rqstp, struct nfsd4_open *open)
 {
 	clientid_t *clientid = &open->op_clientid;
 	struct nfs4_client *clp = NULL;
 	unsigned int strhashval;
 	struct nfs4_stateowner *sop = NULL;
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 
 	if (!check_name(open->op_owner))
 		return nfserr_inval;
@@ -2222,6 +2223,9 @@ nfsd4_process_open1(struct nfsd4_open *open)
 			return nfserr_expired;
 		goto renew;
 	}
+	/* When sessions are used, skip open sequenceid processing */
+	if (nfsd4_has_session(&resp->cstate))
+		goto renew;
 	if (!sop->so_confirmed) {
 		/* Replace unconfirmed owners without checking for replay. */
 		clp = sop->so_client;
@@ -2499,6 +2503,7 @@ out:
 __be32
 nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_open *open)
 {
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfs4_file *fp = NULL;
 	struct inode *ino = current_fh->fh_dentry->d_inode;
 	struct nfs4_stateid *stp = NULL;
@@ -2557,9 +2562,14 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 			release_open_stateid(stp);
 			goto out;
 		}
+		if (nfsd4_has_session(&resp->cstate))
+			update_stateid(&stp->st_stateid);
 	}
 	memcpy(&open->op_stateid, &stp->st_stateid, sizeof(stateid_t));
 
+	if (nfsd4_has_session(&resp->cstate))
+		open->op_stateowner->so_confirmed = 1;
+
 	/*
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
@@ -2580,7 +2590,8 @@ out:
 	* To finish the open response, we just need to set the rflags.
 	*/
 	open->op_rflags = NFS4_OPEN_RESULT_LOCKTYPE_POSIX;
-	if (!open->op_stateowner->so_confirmed)
+	if (!open->op_stateowner->so_confirmed &&
+	    !nfsd4_has_session(&resp->cstate))
 		open->op_rflags |= NFS4_OPEN_RESULT_CONFIRM;
 
 	return status;
@@ -2797,8 +2808,15 @@ grace_disallows_io(struct inode *inode)
 	return locks_in_grace() && mandatory_lock(inode);
 }
 
-static int check_stateid_generation(stateid_t *in, stateid_t *ref)
+static int check_stateid_generation(stateid_t *in, stateid_t *ref, int flags)
 {
+	/*
+	 * When sessions are used the stateid generation number is ignored
+	 * when it is zero.
+	 */
+	if ((flags & HAS_SESSION) && in->si_generation == 0)
+		goto out;
+
 	/* If the client sends us a stateid from the future, it's buggy: */
 	if (in->si_generation > ref->si_generation)
 		return nfserr_bad_stateid;
@@ -2814,6 +2832,7 @@ static int check_stateid_generation(stateid_t *in, stateid_t *ref)
 	 */
 	if (in->si_generation < ref->si_generation)
 		return nfserr_old_stateid;
+out:
 	return nfs_ok;
 }
 
@@ -2851,7 +2870,8 @@ nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int fl
 		dp = find_delegation_stateid(ino, stateid);
 		if (!dp)
 			goto out;
-		status = check_stateid_generation(stateid, &dp->dl_stateid);
+		status = check_stateid_generation(stateid, &dp->dl_stateid,
+						  flags);
 		if (status)
 			goto out;
 		status = nfs4_check_delegmode(dp, flags);
@@ -2868,7 +2888,8 @@ nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int fl
 			goto out;
 		if (!stp->st_stateowner->so_confirmed)
 			goto out;
-		status = check_stateid_generation(stateid, &stp->st_stateid);
+		status = check_stateid_generation(stateid, &stp->st_stateid,
+						  flags);
 		if (status)
 			goto out;
 		status = nfs4_check_openmode(stp, flags);
@@ -2971,7 +2992,7 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
 	*  For the moment, we ignore the possibility of 
 	*  generation number wraparound.
 	*/
-	if (seqid != sop->so_seqid)
+	if (!(flags & HAS_SESSION) && seqid != sop->so_seqid)
 		goto check_replay;
 
 	if (sop->so_confirmed && flags & CONFIRM) {
@@ -2984,7 +3005,7 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
 				" confirmed yet!\n");
 		return nfserr_bad_stateid;
 	}
-	status = check_stateid_generation(stateid, &stp->st_stateid);
+	status = check_stateid_generation(stateid, &stp->st_stateid, flags);
 	if (status)
 		return status;
 	renew_client(sop->so_client);
@@ -3080,6 +3101,7 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
 	__be32 status;
 	struct nfs4_stateid *stp;
 	unsigned int share_access;
+	int flags = OPEN_STATE;
 
 	dprintk("NFSD: nfsd4_open_downgrade on file %.*s\n", 
 			(int)cstate->current_fh.fh_dentry->d_name.len,
@@ -3089,11 +3111,13 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
 			|| !deny_valid(od->od_share_deny))
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 					od->od_seqid,
 					&od->od_stateid, 
-					OPEN_STATE,
+					flags,
 					&od->od_stateowner, &stp, NULL)))
 		goto out; 
 
@@ -3136,17 +3160,20 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 {
 	__be32 status;
 	struct nfs4_stateid *stp;
+	int flags = OPEN_STATE | CLOSE_STATE;
 
 	dprintk("NFSD: nfsd4_close on file %.*s\n", 
 			(int)cstate->current_fh.fh_dentry->d_name.len,
 			cstate->current_fh.fh_dentry->d_name.name);
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	/* check close_lru for replay */
 	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 					close->cl_seqid,
 					&close->cl_stateid, 
-					OPEN_STATE | CLOSE_STATE,
+					flags,
 					&close->cl_stateowner, &stp, NULL)))
 		goto out; 
 	status = nfs_ok;
@@ -3179,11 +3206,14 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	stateid_t *stateid = &dr->dr_stateid;
 	struct inode *inode;
 	__be32 status;
+	int flags = 0;
 
 	if ((status = fh_verify(rqstp, &cstate->current_fh, S_IFREG, 0)))
 		return status;
 	inode = cstate->current_fh.fh_dentry->d_inode;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 	status = nfserr_bad_stateid;
 	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
@@ -3197,7 +3227,7 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	dp = find_delegation_stateid(inode, stateid);
 	if (!dp)
 		goto out;
-	status = check_stateid_generation(stateid, &dp->dl_stateid);
+	status = check_stateid_generation(stateid, &dp->dl_stateid, flags);
 	if (status)
 		goto out;
 	renew_client(dp->dl_client);
@@ -3459,7 +3489,7 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	__be32 status = 0;
 	unsigned int strhashval;
 	unsigned int cmd;
-	int err;
+	int err, flags = 0;
 
 	dprintk("NFSD: nfsd4_lock: start=%Ld length=%Ld\n",
 		(long long) lock->lk_offset,
@@ -3489,11 +3519,15 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		if (STALE_CLIENTID(&lock->lk_new_clientid))
 			goto out;
 
+		flags = OPEN_STATE;
+		if (nfsd4_has_session(cstate))
+			flags |= HAS_SESSION;
+
 		/* validate and update open stateid and open seqid */
 		status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 				        lock->lk_new_open_seqid,
 		                        &lock->lk_new_open_stateid,
-					OPEN_STATE,
+					flags,
 		                        &lock->lk_replay_owner, &open_stp,
 					lock);
 		if (status)
@@ -3516,11 +3550,15 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		if (lock_stp == NULL)
 			goto out;
 	} else {
+		flags = LOCK_STATE;
+		if (nfsd4_has_session(cstate))
+			flags |= HAS_SESSION;
+
 		/* lock (lock owner + lock stateid) already exists */
 		status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 				       lock->lk_old_lock_seqid, 
 				       &lock->lk_old_lock_stateid, 
-				       LOCK_STATE,
+				       flags,
 				       &lock->lk_replay_owner, &lock_stp, lock);
 		if (status)
 			goto out;
@@ -3702,7 +3740,7 @@ nfsd4_locku(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	struct file *filp = NULL;
 	struct file_lock file_lock;
 	__be32 status;
-	int err;
+	int err, flags = LOCK_STATE;
 						        
 	dprintk("NFSD: nfsd4_locku: start=%Ld length=%Ld\n",
 		(long long) locku->lu_offset,
@@ -3711,12 +3749,14 @@ nfsd4_locku(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (check_lock_length(locku->lu_offset, locku->lu_length))
 		 return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		flags |= HAS_SESSION;
 	nfs4_lock_state();
 									        
 	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
 					locku->lu_seqid, 
 					&locku->lu_stateid, 
-					LOCK_STATE,
+					flags,
 					&locku->lu_stateowner, &stp, NULL)))
 		goto out;
 
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 5720aab..a2682e8 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3206,7 +3206,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
 	BUG_ON(iov->iov_len > PAGE_SIZE);
 #ifdef CONFIG_NFSD_V4_1
-	if (resp->cstate.slot != NULL) {
+	if (nfsd4_has_session(&resp->cstate)) {
 		if (resp->cstate.status == nfserr_replay_cache &&
 				!nfsd4_no_page_in_cache(resp)) {
 			iov->iov_len = resp->cstate.iovlen;
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 47c7836..302557d 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -323,6 +323,7 @@ struct nfs4_stateid {
 };
 
 /* flags for preprocess_seqid_op() */
+#define HAS_SESSION             0x00000001
 #define CONFIRM                 0x00000002
 #define OPEN_STATE              0x00000004
 #define LOCK_STATE              0x00000008
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 37a7c51..aafbfdc 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -55,6 +55,11 @@ struct nfsd4_compound_state {
 	u32			status;
 };
 
+static inline bool nfsd4_has_session(struct nfsd4_compound_state *cs)
+{
+	return cs->slot != NULL;
+}
+
 struct nfsd4_change_info {
 	u32		atomic;
 	u32		before_ctime_sec;
@@ -540,7 +545,8 @@ extern __be32 nfsd4_destroy_session(struct svc_rqst *,
 		struct nfsd4_compound_state *,
 		struct nfsd4_destroy_session *);
 #endif /* CONFIG_NFSD_V4_1 */
-extern __be32 nfsd4_process_open1(struct nfsd4_open *open);
+extern __be32 nfsd4_process_open1(struct svc_rqst *rqstp,
+		struct nfsd4_open *open);
 extern __be32 nfsd4_process_open2(struct svc_rqst *rqstp,
 		struct svc_fh *current_fh, struct nfsd4_open *open);
 extern __be32 nfsd4_open_confirm(struct svc_rqst *rqstp,
-- 
1.6.2.1

Re: [PATCH v2 27/47] nfsd41: stateid handling

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:33:31AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> When sessions are used, stateful operation sequenceid and stateid handling
> are not used. When sessions are used,  on the first open set the seqid to 1,
> mark state confirmed and skip seqid processing.
> 
> When sessionas are used the stateid generation number is ignored when it is zero
> whereas without sessions bad_stateid or stale stateid is returned.
> 
> Add flags to propagate session use to all stateful ops and down to
> check_stateid_generation.
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> Signed-off-by: Andy Adamson <andros@netapp.com>
> [nfsd4_has_session should return a boolean, not u32]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4proc.c         |   17 ++++++++--
>  fs/nfsd/nfs4state.c        |   70 ++++++++++++++++++++++++++++++++++---------
>  fs/nfsd/nfs4xdr.c          |    2 +-
>  include/linux/nfsd/state.h |    1 +
>  include/linux/nfsd/xdr4.h  |    8 ++++-
>  5 files changed, 77 insertions(+), 21 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index a273023..1d4b2b5 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -179,7 +179,7 @@ nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	nfs4_lock_state();
>  
>  	/* check seqid for replay. set nfs4_owner */
> -	status = nfsd4_process_open1(open);
> +	status = nfsd4_process_open1(rqstp, open);

Seems all your using is the cstate--maybe pass that instead?

>  	if (status == nfserr_replay_me) {
>  		struct nfs4_replay *rp = &open->op_stateowner->so_replay;
>  		fh_put(&cstate->current_fh);
> @@ -504,6 +504,7 @@ nfsd4_read(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	   struct nfsd4_read *read)
>  {
>  	__be32 status;
> +	int flags = RD_STATE;
>  
>  	/* no need to check permission - this will be done in nfsd_read() */
>  
> @@ -511,11 +512,13 @@ nfsd4_read(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	if (read->rd_offset >= OFFSET_MAX)
>  		return nfserr_inval;
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	nfs4_lock_state();
>  	/* check stateid */
>  	if ((status = nfs4_preprocess_stateid_op(&cstate->current_fh,

You could pass the cstate to preprocess_stateid_op instead of
current_fh, and let it do the nfsd4_has_session check instead of making
all the callers do it.

--b.

>  				&read->rd_stateid,
> -				RD_STATE, &read->rd_filp))) {
> +				flags, &read->rd_filp))) {
>  		dprintk("NFSD: nfsd4_read: couldn't process stateid!\n");
>  		goto out;
>  	}
> @@ -643,11 +646,14 @@ nfsd4_setattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	      struct nfsd4_setattr *setattr)
>  {
>  	__be32 status = nfs_ok;
> +	int flags = WR_STATE;
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
>  		nfs4_lock_state();
>  		status = nfs4_preprocess_stateid_op(&cstate->current_fh,
> -			&setattr->sa_stateid, WR_STATE, NULL);
> +			&setattr->sa_stateid, flags, NULL);
>  		nfs4_unlock_state();
>  		if (status) {
>  			dprintk("NFSD: nfsd4_setattr: couldn't process stateid!\n");
> @@ -679,15 +685,18 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	u32 *p;
>  	__be32 status = nfs_ok;
>  	unsigned long cnt;
> +	int flags = WR_STATE;
>  
>  	/* no need to check permission - this will be done in nfsd_write() */
>  
>  	if (write->wr_offset >= OFFSET_MAX)
>  		return nfserr_inval;
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	nfs4_lock_state();
>  	status = nfs4_preprocess_stateid_op(&cstate->current_fh, stateid,
> -					WR_STATE, &filp);
> +					    flags, &filp);
>  	if (filp)
>  		get_file(filp);
>  	nfs4_unlock_state();
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 9c93f96..bf5b214 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -2199,12 +2199,13 @@ static struct lock_manager_operations nfsd_lease_mng_ops = {
>  
>  
>  __be32
> -nfsd4_process_open1(struct nfsd4_open *open)
> +nfsd4_process_open1(struct svc_rqst *rqstp, struct nfsd4_open *open)
>  {
>  	clientid_t *clientid = &open->op_clientid;
>  	struct nfs4_client *clp = NULL;
>  	unsigned int strhashval;
>  	struct nfs4_stateowner *sop = NULL;
> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>  
>  	if (!check_name(open->op_owner))
>  		return nfserr_inval;
> @@ -2222,6 +2223,9 @@ nfsd4_process_open1(struct nfsd4_open *open)
>  			return nfserr_expired;
>  		goto renew;
>  	}
> +	/* When sessions are used, skip open sequenceid processing */
> +	if (nfsd4_has_session(&resp->cstate))
> +		goto renew;
>  	if (!sop->so_confirmed) {
>  		/* Replace unconfirmed owners without checking for replay. */
>  		clp = sop->so_client;
> @@ -2499,6 +2503,7 @@ out:
>  __be32
>  nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_open *open)
>  {
> +	struct nfsd4_compoundres *resp = rqstp->rq_resp;
>  	struct nfs4_file *fp = NULL;
>  	struct inode *ino = current_fh->fh_dentry->d_inode;
>  	struct nfs4_stateid *stp = NULL;
> @@ -2557,9 +2562,14 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
>  			release_open_stateid(stp);
>  			goto out;
>  		}
> +		if (nfsd4_has_session(&resp->cstate))
> +			update_stateid(&stp->st_stateid);
>  	}
>  	memcpy(&open->op_stateid, &stp->st_stateid, sizeof(stateid_t));
>  
> +	if (nfsd4_has_session(&resp->cstate))
> +		open->op_stateowner->so_confirmed = 1;
> +
>  	/*
>  	* Attempt to hand out a delegation. No error return, because the
>  	* OPEN succeeds even if we fail.
> @@ -2580,7 +2590,8 @@ out:
>  	* To finish the open response, we just need to set the rflags.
>  	*/
>  	open->op_rflags = NFS4_OPEN_RESULT_LOCKTYPE_POSIX;
> -	if (!open->op_stateowner->so_confirmed)
> +	if (!open->op_stateowner->so_confirmed &&
> +	    !nfsd4_has_session(&resp->cstate))
>  		open->op_rflags |= NFS4_OPEN_RESULT_CONFIRM;
>  
>  	return status;
> @@ -2797,8 +2808,15 @@ grace_disallows_io(struct inode *inode)
>  	return locks_in_grace() && mandatory_lock(inode);
>  }
>  
> -static int check_stateid_generation(stateid_t *in, stateid_t *ref)
> +static int check_stateid_generation(stateid_t *in, stateid_t *ref, int flags)
>  {
> +	/*
> +	 * When sessions are used the stateid generation number is ignored
> +	 * when it is zero.
> +	 */
> +	if ((flags & HAS_SESSION) && in->si_generation == 0)
> +		goto out;
> +
>  	/* If the client sends us a stateid from the future, it's buggy: */
>  	if (in->si_generation > ref->si_generation)
>  		return nfserr_bad_stateid;
> @@ -2814,6 +2832,7 @@ static int check_stateid_generation(stateid_t *in, stateid_t *ref)
>  	 */
>  	if (in->si_generation < ref->si_generation)
>  		return nfserr_old_stateid;
> +out:
>  	return nfs_ok;
>  }
>  
> @@ -2851,7 +2870,8 @@ nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int fl
>  		dp = find_delegation_stateid(ino, stateid);
>  		if (!dp)
>  			goto out;
> -		status = check_stateid_generation(stateid, &dp->dl_stateid);
> +		status = check_stateid_generation(stateid, &dp->dl_stateid,
> +						  flags);
>  		if (status)
>  			goto out;
>  		status = nfs4_check_delegmode(dp, flags);
> @@ -2868,7 +2888,8 @@ nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int fl
>  			goto out;
>  		if (!stp->st_stateowner->so_confirmed)
>  			goto out;
> -		status = check_stateid_generation(stateid, &stp->st_stateid);
> +		status = check_stateid_generation(stateid, &stp->st_stateid,
> +						  flags);
>  		if (status)
>  			goto out;
>  		status = nfs4_check_openmode(stp, flags);
> @@ -2971,7 +2992,7 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
>  	*  For the moment, we ignore the possibility of 
>  	*  generation number wraparound.
>  	*/
> -	if (seqid != sop->so_seqid)
> +	if (!(flags & HAS_SESSION) && seqid != sop->so_seqid)
>  		goto check_replay;
>  
>  	if (sop->so_confirmed && flags & CONFIRM) {
> @@ -2984,7 +3005,7 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
>  				" confirmed yet!\n");
>  		return nfserr_bad_stateid;
>  	}
> -	status = check_stateid_generation(stateid, &stp->st_stateid);
> +	status = check_stateid_generation(stateid, &stp->st_stateid, flags);
>  	if (status)
>  		return status;
>  	renew_client(sop->so_client);
> @@ -3080,6 +3101,7 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
>  	__be32 status;
>  	struct nfs4_stateid *stp;
>  	unsigned int share_access;
> +	int flags = OPEN_STATE;
>  
>  	dprintk("NFSD: nfsd4_open_downgrade on file %.*s\n", 
>  			(int)cstate->current_fh.fh_dentry->d_name.len,
> @@ -3089,11 +3111,13 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
>  			|| !deny_valid(od->od_share_deny))
>  		return nfserr_inval;
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	nfs4_lock_state();
>  	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
>  					od->od_seqid,
>  					&od->od_stateid, 
> -					OPEN_STATE,
> +					flags,
>  					&od->od_stateowner, &stp, NULL)))
>  		goto out; 
>  
> @@ -3136,17 +3160,20 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  {
>  	__be32 status;
>  	struct nfs4_stateid *stp;
> +	int flags = OPEN_STATE | CLOSE_STATE;
>  
>  	dprintk("NFSD: nfsd4_close on file %.*s\n", 
>  			(int)cstate->current_fh.fh_dentry->d_name.len,
>  			cstate->current_fh.fh_dentry->d_name.name);
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	nfs4_lock_state();
>  	/* check close_lru for replay */
>  	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
>  					close->cl_seqid,
>  					&close->cl_stateid, 
> -					OPEN_STATE | CLOSE_STATE,
> +					flags,
>  					&close->cl_stateowner, &stp, NULL)))
>  		goto out; 
>  	status = nfs_ok;
> @@ -3179,11 +3206,14 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	stateid_t *stateid = &dr->dr_stateid;
>  	struct inode *inode;
>  	__be32 status;
> +	int flags = 0;
>  
>  	if ((status = fh_verify(rqstp, &cstate->current_fh, S_IFREG, 0)))
>  		return status;
>  	inode = cstate->current_fh.fh_dentry->d_inode;
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	nfs4_lock_state();
>  	status = nfserr_bad_stateid;
>  	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
> @@ -3197,7 +3227,7 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	dp = find_delegation_stateid(inode, stateid);
>  	if (!dp)
>  		goto out;
> -	status = check_stateid_generation(stateid, &dp->dl_stateid);
> +	status = check_stateid_generation(stateid, &dp->dl_stateid, flags);
>  	if (status)
>  		goto out;
>  	renew_client(dp->dl_client);
> @@ -3459,7 +3489,7 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	__be32 status = 0;
>  	unsigned int strhashval;
>  	unsigned int cmd;
> -	int err;
> +	int err, flags = 0;
>  
>  	dprintk("NFSD: nfsd4_lock: start=%Ld length=%Ld\n",
>  		(long long) lock->lk_offset,
> @@ -3489,11 +3519,15 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  		if (STALE_CLIENTID(&lock->lk_new_clientid))
>  			goto out;
>  
> +		flags = OPEN_STATE;
> +		if (nfsd4_has_session(cstate))
> +			flags |= HAS_SESSION;
> +
>  		/* validate and update open stateid and open seqid */
>  		status = nfs4_preprocess_seqid_op(&cstate->current_fh,
>  				        lock->lk_new_open_seqid,
>  		                        &lock->lk_new_open_stateid,
> -					OPEN_STATE,
> +					flags,
>  		                        &lock->lk_replay_owner, &open_stp,
>  					lock);
>  		if (status)
> @@ -3516,11 +3550,15 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  		if (lock_stp == NULL)
>  			goto out;
>  	} else {
> +		flags = LOCK_STATE;
> +		if (nfsd4_has_session(cstate))
> +			flags |= HAS_SESSION;
> +
>  		/* lock (lock owner + lock stateid) already exists */
>  		status = nfs4_preprocess_seqid_op(&cstate->current_fh,
>  				       lock->lk_old_lock_seqid, 
>  				       &lock->lk_old_lock_stateid, 
> -				       LOCK_STATE,
> +				       flags,
>  				       &lock->lk_replay_owner, &lock_stp, lock);
>  		if (status)
>  			goto out;
> @@ -3702,7 +3740,7 @@ nfsd4_locku(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	struct file *filp = NULL;
>  	struct file_lock file_lock;
>  	__be32 status;
> -	int err;
> +	int err, flags = LOCK_STATE;
>  						        
>  	dprintk("NFSD: nfsd4_locku: start=%Ld length=%Ld\n",
>  		(long long) locku->lu_offset,
> @@ -3711,12 +3749,14 @@ nfsd4_locku(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
>  	if (check_lock_length(locku->lu_offset, locku->lu_length))
>  		 return nfserr_inval;
>  
> +	if (nfsd4_has_session(cstate))
> +		flags |= HAS_SESSION;
>  	nfs4_lock_state();
>  									        
>  	if ((status = nfs4_preprocess_seqid_op(&cstate->current_fh,
>  					locku->lu_seqid, 
>  					&locku->lu_stateid, 
> -					LOCK_STATE,
> +					flags,
>  					&locku->lu_stateowner, &stp, NULL)))
>  		goto out;
>  
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index 5720aab..a2682e8 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -3206,7 +3206,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
>  	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
>  	BUG_ON(iov->iov_len > PAGE_SIZE);
>  #ifdef CONFIG_NFSD_V4_1
> -	if (resp->cstate.slot != NULL) {
> +	if (nfsd4_has_session(&resp->cstate)) {
>  		if (resp->cstate.status == nfserr_replay_cache &&
>  				!nfsd4_no_page_in_cache(resp)) {
>  			iov->iov_len = resp->cstate.iovlen;
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index 47c7836..302557d 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -323,6 +323,7 @@ struct nfs4_stateid {
>  };
>  
>  /* flags for preprocess_seqid_op() */
> +#define HAS_SESSION             0x00000001
>  #define CONFIRM                 0x00000002
>  #define OPEN_STATE              0x00000004
>  #define LOCK_STATE              0x00000008
> diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
> index 37a7c51..aafbfdc 100644
> --- a/include/linux/nfsd/xdr4.h
> +++ b/include/linux/nfsd/xdr4.h
> @@ -55,6 +55,11 @@ struct nfsd4_compound_state {
>  	u32			status;
>  };
>  
> +static inline bool nfsd4_has_session(struct nfsd4_compound_state *cs)
> +{
> +	return cs->slot != NULL;
> +}
> +
>  struct nfsd4_change_info {
>  	u32		atomic;
>  	u32		before_ctime_sec;
> @@ -540,7 +545,8 @@ extern __be32 nfsd4_destroy_session(struct svc_rqst *,
>  		struct nfsd4_compound_state *,
>  		struct nfsd4_destroy_session *);
>  #endif /* CONFIG_NFSD_V4_1 */
> -extern __be32 nfsd4_process_open1(struct nfsd4_open *open);
> +extern __be32 nfsd4_process_open1(struct svc_rqst *rqstp,
> +		struct nfsd4_open *open);
>  extern __be32 nfsd4_process_open2(struct svc_rqst *rqstp,
>  		struct svc_fh *current_fh, struct nfsd4_open *open);
>  extern __be32 nfsd4_open_confirm(struct svc_rqst *rqstp,
> -- 
> 1.6.2.1
> 

[PATCH v2 28/47] nfsd41: check encode size for sessions maxresponse cached

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Calculate the space the compound response has taken after encoding the current
operation.

pad: add on 8 bytes for the next operation's op_code and status so that
there is room to cache a failure on the next operation.

Compare this length to the session se_fmaxresp_cached and return
nfserr_rep_too_big_to_cache if the length is too large.

Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
will be at least a page and will therefore hold the xdr_buf head.

Signed-off-by: Andy Adamson <andros@netapp.com>
[nfsd41: non-page DRC for solo sequence responses]
[fixed nfsd4_check_drc_limit cosmetics]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 58 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a2682e8..52ca833 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3089,6 +3089,61 @@ static nfsd4_enc nfsd4_enc_ops[] = {
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * Calculate the total amount of memory that the compound response has taken
+ * after encoding the current operation.
+ *
+ * pad: add on 8 bytes for the next operation's op_code and status so that
+ * there is room to cache a failure on the next operation.
+ *
+ * Compare this length to the session se_fmaxresp_cached.
+ *
+ * Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
+ * will be at least a page and will therefore hold the xdr_buf head.
+ */
+static int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
+{
+	int status = 0;
+	struct xdr_buf *xb = &resp->rqstp->rq_res;
+	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
+	struct nfsd4_session *session = NULL;
+	struct nfsd4_slot *slot = resp->cstate.slot;
+	u32 length, tlen = 0, pad = 8;
+
+	if (!nfsd4_has_session(&resp->cstate))
+		return status;
+
+	session = slot->sl_session;
+	if (session == NULL || slot->sl_cache_entry.ce_cachethis == 0)
+		return status;
+
+	if (resp->opcnt >= args->opcnt)
+		pad = 0; /* this is the last operation */
+
+	if (xb->page_len == 0) {
+		length = (char *)resp->p - (char *)xb->head[0].iov_base + pad;
+	} else {
+		if (xb->tail[0].iov_base && xb->tail[0].iov_len > 0)
+			tlen = (char *)resp->p - (char *)xb->tail[0].iov_base;
+
+		length = xb->head[0].iov_len + xb->page_len + tlen + pad;
+	}
+	dprintk("%s length %u, xb->page_len %u tlen %u pad %u\n", __func__,
+		length, xb->page_len, tlen, pad);
+
+	if (length <= session->se_fmaxresp_cached)
+		return status;
+	else
+		return nfserr_rep_too_big_to_cache;
+}
+#else /* CONFIG_NFSD_V4_1 */
+static inline int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
+{
+	return 0;
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
@@ -3105,6 +3160,9 @@ nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 	BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
 	       !nfsd4_enc_ops[op->opnum]);
 	op->status = nfsd4_enc_ops[op->opnum](resp, op->status, &op->u);
+	/* nfsd4_check_drc_limit guarantees enough room for error status */
+	if (!op->status && nfsd4_check_drc_limit(resp))
+		op->status = nfserr_rep_too_big_to_cache;
 status:
 	/*
 	 * Note: We write the status directly, instead of using WRITE32(),
-- 
1.6.2.1

Re: [PATCH v2 28/47] nfsd41: check encode size for sessions maxresponse cached

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:33:38AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Calculate the space the compound response has taken after encoding the current
> operation.
> 
> pad: add on 8 bytes for the next operation's op_code and status so that
> there is room to cache a failure on the next operation.

Looks like setattr always has a bitmap regardless of status, so that
should be at least 12.  There might be some other odd case like that
too--someone should look through the xdr and check.

> Compare this length to the session se_fmaxresp_cached and return
> nfserr_rep_too_big_to_cache if the length is too large.
> 
> Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
> will be at least a page and will therefore hold the xdr_buf head.
> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> [nfsd41: non-page DRC for solo sequence responses]
> [fixed nfsd4_check_drc_limit cosmetics]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4xdr.c |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 58 insertions(+), 0 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
> index a2682e8..52ca833 100644
> --- a/fs/nfsd/nfs4xdr.c
> +++ b/fs/nfsd/nfs4xdr.c
> @@ -3089,6 +3089,61 @@ static nfsd4_enc nfsd4_enc_ops[] = {
>  #endif /* CONFIG_NFSD_V4_1 */
>  };
>  
> +#if defined(CONFIG_NFSD_V4_1)
> +/*
> + * Calculate the total amount of memory that the compound response has taken
> + * after encoding the current operation.
> + *
> + * pad: add on 8 bytes for the next operation's op_code and status so that
> + * there is room to cache a failure on the next operation.
> + *
> + * Compare this length to the session se_fmaxresp_cached.
> + *
> + * Our se_fmaxresp_cached will always be a multiple of PAGE_SIZE, and so
> + * will be at least a page and will therefore hold the xdr_buf head.
> + */
> +static int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
> +{
> +	int status = 0;

Note that status and the return value should be __be32.

Ditto for anything holding an nfs error or the server side.  I haven't
been checking for this as I've been reading these.

Probably we should be running sparse regularly (install it, then build
the kernel with 'make C=1' or 'C=2')--I think it checks for this.

> +	struct xdr_buf *xb = &resp->rqstp->rq_res;
> +	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
> +	struct nfsd4_session *session = NULL;
> +	struct nfsd4_slot *slot = resp->cstate.slot;
> +	u32 length, tlen = 0, pad = 8;
> +
> +	if (!nfsd4_has_session(&resp->cstate))
> +		return status;
> +
> +	session = slot->sl_session;
> +	if (session == NULL || slot->sl_cache_entry.ce_cachethis == 0)
> +		return status;
> +
> +	if (resp->opcnt >= args->opcnt)
> +		pad = 0; /* this is the last operation */
> +
> +	if (xb->page_len == 0) {
> +		length = (char *)resp->p - (char *)xb->head[0].iov_base + pad;
> +	} else {
> +		if (xb->tail[0].iov_base && xb->tail[0].iov_len > 0)
> +			tlen = (char *)resp->p - (char *)xb->tail[0].iov_base;
> +
> +		length = xb->head[0].iov_len + xb->page_len + tlen + pad;
> +	}
> +	dprintk("%s length %u, xb->page_len %u tlen %u pad %u\n", __func__,
> +		length, xb->page_len, tlen, pad);
> +
> +	if (length <= session->se_fmaxresp_cached)
> +		return status;
> +	else
> +		return nfserr_rep_too_big_to_cache;
> +}
> +#else /* CONFIG_NFSD_V4_1 */
> +static inline int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
> +{
> +	return 0;
> +}
> +#endif /* CONFIG_NFSD_V4_1 */
> +
>  void
>  nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
>  {
> @@ -3105,6 +3160,9 @@ nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
>  	BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
>  	       !nfsd4_enc_ops[op->opnum]);
>  	op->status = nfsd4_enc_ops[op->opnum](resp, op->status, &op->u);
> +	/* nfsd4_check_drc_limit guarantees enough room for error status */
> +	if (!op->status && nfsd4_check_drc_limit(resp))

Hm, but note you aren't actually using the return value from
nfsd4_check_drc_limit().

> +		op->status = nfserr_rep_too_big_to_cache;

This means you can end up writing the error after you've already encoded
a succesful reply.  That doesn't work--the result will be invalid xdr.

I think there's no alternative but to estimate the size of the result
before actually performing the operation, because once we decide to
process an operation which the client has asked us to cache, we're
committed to caching the result.

Maybe add a field to struct nfsd4_operation consisting of a function
like

	estimate_reply_size(void *arg)

which does the calculation for each op.  Or maybe there's some clever
way to avoid duplicating a lot of information that's already implicit in
the xdr-encoding routines.

Am I missing something?

--b.

>  status:
>  	/*
>  	 * Note: We write the status directly, instead of using WRITE32(),
> -- 
> 1.6.2.1
> 

[PATCH v2 29/47] nfsd41: clientid handling

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Extract the clientid from sessionid to set the op_clientid on open.
Verify that the clid for other stateful ops is zero for minorversion != 0
Do all other checks for stateful ops without sessions.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
Signed-off-by: Andy Adamson <andros@netapp.com>
[fixed whitespace indent]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c  |   12 ++++++++++++
 fs/nfsd/nfs4state.c |   14 ++++++++------
 fs/nfsd/nfs4xdr.c   |    9 +++++++++
 3 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 1d4b2b5..0076bb1 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -162,6 +162,15 @@ do_open_fhandle(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_
 	return status;
 }
 
+static void
+copy_clientid(clientid_t *clid, struct nfsd4_session *session)
+{
+	struct nfsd4_sessionid *sid =
+			(struct nfsd4_sessionid *)session->se_sessionid.data;
+
+	clid->cl_boot = sid->clientid.cl_boot;
+	clid->cl_id = sid->clientid.cl_id;
+}
 
 static __be32
 nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
@@ -176,6 +185,9 @@ nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (open->op_create && open->op_claim_type != NFS4_OPEN_CLAIM_NULL)
 		return nfserr_inval;
 
+	if (nfsd4_has_session(cstate))
+		copy_clientid(&open->op_clientid, cstate->slot->sl_session);
+
 	nfs4_lock_state();
 
 	/* check seqid for replay. set nfs4_owner */
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index bf5b214..00ad1c2 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -616,8 +616,8 @@ STALE_CLIENTID(clientid_t *clid)
 {
 	if (clid->cl_boot == boot_time)
 		return 0;
-	dprintk("NFSD stale clientid (%08x/%08x)\n", 
-			clid->cl_boot, clid->cl_id);
+	dprintk("NFSD stale clientid (%08x/%08x) boot_time %08lx\n",
+		clid->cl_boot, clid->cl_id, boot_time);
 	return 1;
 }
 
@@ -2968,8 +2968,9 @@ nfs4_preprocess_seqid_op(struct svc_fh *current_fh, u32 seqid, stateid_t *statei
 		if (lock->lk_is_new) {
 			if (!sop->so_is_open_owner)
 				return nfserr_bad_stateid;
-			if (!same_clid(&clp->cl_clientid, lockclid))
-			       return nfserr_bad_stateid;
+			if (!(flags & HAS_SESSION) &&
+			    !same_clid(&clp->cl_clientid, lockclid))
+				return nfserr_bad_stateid;
 			/* stp is the open stateid */
 			status = nfs4_check_openmode(stp, lkflg);
 			if (status)
@@ -3516,7 +3517,8 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		struct nfs4_file *fp;
 		
 		status = nfserr_stale_clientid;
-		if (STALE_CLIENTID(&lock->lk_new_clientid))
+		if (!nfsd4_has_session(cstate) &&
+		    STALE_CLIENTID(&lock->lk_new_clientid))
 			goto out;
 
 		flags = OPEN_STATE;
@@ -3678,7 +3680,7 @@ nfsd4_lockt(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	nfs4_lock_state();
 
 	status = nfserr_stale_clientid;
-	if (STALE_CLIENTID(&lockt->lt_clientid))
+	if (!nfsd4_has_session(cstate) && STALE_CLIENTID(&lockt->lt_clientid))
 		goto out;
 
 	if ((status = fh_verify(rqstp, &cstate->current_fh, S_IFREG, 0))) {
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 52ca833..a1af3a3 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -189,6 +189,11 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	return p;
 }
 
+static int zero_clientid(clientid_t *clid)
+{
+	return (clid->cl_boot == 0) && (clid->cl_id == 0);
+}
+
 static int
 defer_free(struct nfsd4_compoundargs *argp,
 		void (*release)(const void *), void *p)
@@ -584,6 +589,8 @@ nfsd4_decode_lockt(struct nfsd4_compoundargs *argp, struct nfsd4_lockt *lockt)
 	READ_BUF(lockt->lt_owner.len);
 	READMEM(lockt->lt_owner.data, lockt->lt_owner.len);
 
+	if (argp->minorversion && !zero_clientid(&lockt->lt_clientid))
+		return nfserr_inval;
 	DECODE_TAIL;
 }
 
@@ -994,6 +1001,8 @@ nfsd4_decode_release_lockowner(struct nfsd4_compoundargs *argp, struct nfsd4_rel
 	READ_BUF(rlockowner->rl_owner.len);
 	READMEM(rlockowner->rl_owner.data, rlockowner->rl_owner.len);
 
+	if (argp->minorversion && !zero_clientid(&rlockowner->rl_clientid))
+		return nfserr_inval;
 	DECODE_TAIL;
 }
 
-- 
1.6.2.1

[PATCH v2 30/47] nfsd41: access_valid

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

For nfs41, the open share flags are used also for
delegation "wants" and "signals".  Check that they are valid.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |    1 +
 fs/nfsd/nfs4state.c       |   20 +++++++++++++++-----
 include/linux/nfsd/xdr4.h |    1 +
 3 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 0076bb1..8b4bf5e 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -901,6 +901,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	resp->tag = args->tag;
 	resp->opcnt = 0;
 	resp->rqstp = rqstp;
+	resp->cstate.minorversion = args->minorversion;
 	resp->cstate.replay_owner = NULL;
 	fh_init(&resp->cstate.current_fh, NFS4_FHSIZE);
 	fh_init(&resp->cstate.save_fh, NFS4_FHSIZE);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 00ad1c2..dbf0d07 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1959,11 +1959,21 @@ find_file(struct inode *ino)
 	return NULL;
 }
 
-static inline int access_valid(u32 x)
+static inline int access_valid(u32 x, u32 minorversion)
 {
-	if (x < NFS4_SHARE_ACCESS_READ)
+	if ((x & NFS4_SHARE_ACCESS_MASK) < NFS4_SHARE_ACCESS_READ)
 		return 0;
-	if (x > NFS4_SHARE_ACCESS_BOTH)
+	if ((x & NFS4_SHARE_ACCESS_MASK) > NFS4_SHARE_ACCESS_BOTH)
+		return 0;
+	x &= ~NFS4_SHARE_ACCESS_MASK;
+	if (minorversion && x) {
+		if ((x & NFS4_SHARE_WANT_MASK) > NFS4_SHARE_WANT_CANCEL)
+			return 0;
+		if ((x & NFS4_SHARE_WHEN_MASK) > NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED)
+			return 0;
+		x &= ~(NFS4_SHARE_WANT_MASK | NFS4_SHARE_WHEN_MASK);
+	}
+	if (x)
 		return 0;
 	return 1;
 }
@@ -2511,7 +2521,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	__be32 status;
 
 	status = nfserr_inval;
-	if (!access_valid(open->op_share_access)
+	if (!access_valid(open->op_share_access, resp->cstate.minorversion)
 			|| !deny_valid(open->op_share_deny))
 		goto out;
 	/*
@@ -3108,7 +3118,7 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
 			(int)cstate->current_fh.fh_dentry->d_name.len,
 			cstate->current_fh.fh_dentry->d_name.name);
 
-	if (!access_valid(od->od_share_access)
+	if (!access_valid(od->od_share_access, cstate->minorversion)
 			|| !deny_valid(od->od_share_deny))
 		return nfserr_inval;
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index aafbfdc..881ff67 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -52,6 +52,7 @@ struct nfsd4_compound_state {
 	struct nfsd4_slot	*slot;
 	__be32			*statp;
 	size_t			iovlen;
+	u32			minorversion;
 	u32			status;
 };
 
-- 
1.6.2.1

[PATCH v2 31/47] nfsd41: add OPEN4_SHARE_ACCESS_WANT nfs4_stateid bmap

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Separate the access bits from the want bits and enable __set_bit to
work correctly with st_access_bmap.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index dbf0d07..8a64981 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1905,7 +1905,8 @@ init_stateid(struct nfs4_stateid *stp, struct nfs4_file *fp, struct nfsd4_open *
 	stp->st_stateid.si_generation = 0;
 	stp->st_access_bmap = 0;
 	stp->st_deny_bmap = 0;
-	__set_bit(open->op_share_access, &stp->st_access_bmap);
+	__set_bit(open->op_share_access & ~NFS4_SHARE_WANT_MASK,
+		  &stp->st_access_bmap);
 	__set_bit(open->op_share_deny, &stp->st_deny_bmap);
 	stp->st_openstp = NULL;
 }
-- 
1.6.2.1

[PATCH v2 32/47] nfsd41: provide support for minor version 1 at rpc level

[Benny Halevy]

From: Marc Eshel <eshel@almaden.ibm.com>

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/nfsd/nfsd.h |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index 3bccec3..d0fe9b4 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -23,7 +23,11 @@
 /*
  * nfsd version
  */
+#if defined(CONFIG_NFSD_V4_1)
+#define NFSD_SUPPORTED_MINOR_VERSION	1
+#else /* CONFIG_NFSD_V4_1 */
 #define NFSD_SUPPORTED_MINOR_VERSION	0
+#endif /* CONFIG_NFSD_V4_1 */
 
 /*
  * Flags for nfsd_permission
-- 
1.6.2.1

[PATCH v2 33/47] nfsd: cleanup nfs4.0 callback encode routines

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Mimic the client and prepare to share the back channel xdr with NFSv4.1.
Bump the number of operations in each encode routine, then backfill the
number of operations.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   24 ++++++++++++++++--------
 1 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 290289b..7129b0c 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -140,8 +140,9 @@ struct nfs4_cb_compound_hdr {
 	int		status;
 	u32		ident;
 	u32		nops;
+	__be32		*nops_p;
 	u32		taglen;
-	char *		tag;
+	char		*tag;
 };
 
 static struct {
@@ -201,7 +202,7 @@ nfs_cb_stat_to_errno(int stat)
  * XDR encode
  */
 
-static int
+static void
 encode_cb_compound_hdr(struct xdr_stream *xdr, struct nfs4_cb_compound_hdr *hdr)
 {
 	__be32 * p;
@@ -210,12 +211,18 @@ encode_cb_compound_hdr(struct xdr_stream *xdr, struct nfs4_cb_compound_hdr *hdr)
 	WRITE32(0);            /* tag length is always 0 */
 	WRITE32(NFS4_MINOR_VERSION);
 	WRITE32(hdr->ident);
+	hdr->nops_p = p;
 	WRITE32(hdr->nops);
-	return 0;
 }
 
-static int
-encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec)
+static void encode_cb_nops(struct nfs4_cb_compound_hdr *hdr)
+{
+	*hdr->nops_p = htonl(hdr->nops);
+}
+
+static void
+encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
+		 struct nfs4_cb_compound_hdr *hdr)
 {
 	__be32 *p;
 	int len = cb_rec->cbr_fh.fh_size;
@@ -227,7 +234,7 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec)
 	WRITE32(cb_rec->cbr_trunc);
 	WRITE32(len);
 	WRITEMEM(&cb_rec->cbr_fh.fh_base, len);
-	return 0;
+	hdr->nops++;
 }
 
 static int
@@ -246,12 +253,13 @@ nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *a
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->cbr_ident,
-		.nops   = 1,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
-	return (encode_cb_recall(&xdr, args));
+	encode_cb_recall(&xdr, args, &hdr);
+	encode_cb_nops(&hdr);
+	return 0;
 }
 
 
-- 
1.6.2.1

[PATCH v2 34/47] sunrpc: add cl_private field to struct rpc_clnt

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Note:  Both the NFSv4.1 client and server use this pointer.

The client uses cl_private to determine the minorversion
(via a struct nfs_client *) to be set in the compound header,
and to know when to generate a SEQUENCE op.
Similarly, the server puts a struct nfs4_callback * in
there for callback compounds' CB_COMPOUND and CB_SEQUENCE.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/sunrpc/clnt.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index c39a210..928c547 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -53,6 +53,7 @@ struct rpc_clnt {
 	char			cl_pathname[30];/* Path in rpc_pipe_fs */
 	struct vfsmount *	cl_vfsmnt;
 	struct dentry *		cl_dentry;	/* inode */
+	void			*cl_private;	/* private data */
 	struct rpc_clnt *	cl_parent;	/* Points to parent of clones */
 	struct rpc_rtt		cl_rtt_default;
 	struct rpc_timeout	cl_timeout_default;
-- 
1.6.2.1

[PATCH v2 35/47] nfsd: minorversion support for the back channel

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Prepare to share backchannel code with NFSv4.1.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c     |    7 ++++++-
 fs/nfsd/nfs4state.c        |    1 +
 include/linux/nfsd/state.h |    3 ++-
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 7129b0c..d8f2631 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -141,6 +141,7 @@ struct nfs4_cb_compound_hdr {
 	u32		ident;
 	u32		nops;
 	__be32		*nops_p;
+	u32		minorversion;
 	u32		taglen;
 	char		*tag;
 };
@@ -209,7 +210,7 @@ encode_cb_compound_hdr(struct xdr_stream *xdr, struct nfs4_cb_compound_hdr *hdr)
 
 	RESERVE_SPACE(16);
 	WRITE32(0);            /* tag length is always 0 */
-	WRITE32(NFS4_MINOR_VERSION);
+	WRITE32(hdr->minorversion);
 	WRITE32(hdr->ident);
 	hdr->nops_p = p;
 	WRITE32(hdr->nops);
@@ -251,8 +252,11 @@ static int
 nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_callback *cb =
+		(struct nfs4_callback *)req->rq_task->tk_client->cl_private;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->cbr_ident,
+		.minorversion = cb->cb_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
@@ -436,6 +440,7 @@ static int do_probe_callback(void *data)
 		goto out_release_client;
 
 	cb->cb_client = client;
+	client->cl_private = cb;
 	atomic_set(&cb->cb_set, 1);
 	put_nfs4_client(clp);
 	return 0;
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 8a64981..189f1c2 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -994,6 +994,7 @@ gen_callback(struct nfs4_client *clp, struct nfsd4_setclientid *se)
 	if ( !(parse_ipv4(se->se_callback_addr_len, se->se_callback_addr_val,
 	                 &cb->cb_addr, &cb->cb_port)))
 		goto out_err;
+	cb->cb_minorversion = 0;
 	cb->cb_prog = se->se_callback_prog;
 	cb->cb_ident = se->se_callback_ident;
 	return;
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index 302557d..b78021a 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -93,7 +93,8 @@ struct nfs4_callback {
 	u32                     cb_addr;
 	unsigned short          cb_port;
 	u32                     cb_prog;
-	u32                     cb_ident;
+	u32			cb_minorversion;
+	u32                     cb_ident;	/* minorversion 0 only */
 	/* RPC client info */
 	atomic_t		cb_set;     /* successful CB_NULL call */
 	struct rpc_clnt *       cb_client;
-- 
1.6.2.1

[PATCH v2 36/47] nfsd41: sunrpc: Added rpc server-side backchannel handling

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

FIXME: bhalevy: write up commit message

Signed-off-by: Mike Sager <sager@netapp.com>
Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>

When the call direction is a reply, copy the xid and call direction into the
req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns
rpc_garbage.

Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 include/linux/sunrpc/clnt.h    |    1 +
 include/linux/sunrpc/svcsock.h |    1 +
 include/linux/sunrpc/xprt.h    |    4 +
 net/sunrpc/clnt.c              |    1 +
 net/sunrpc/svcsock.c           |   81 +++++++++++
 net/sunrpc/xprt.c              |   41 ++++++-
 net/sunrpc/xprtsock.c          |  286 +++++++++++++++++++++++++++++++++++++++-
 7 files changed, 405 insertions(+), 10 deletions(-)

diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 928c547..0b32b00 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -111,6 +111,7 @@ struct rpc_create_args {
 	rpc_authflavor_t	authflavor;
 	unsigned long		flags;
 	char			*client_name;
+	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
 };
 
 /* Values for "flags" field */
diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h
index 483e103..e8fda10 100644
--- a/include/linux/sunrpc/svcsock.h
+++ b/include/linux/sunrpc/svcsock.h
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt	       *sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index 11fc71d..e9ffee3 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -174,6 +174,9 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+#if defined(CONFIG_NFSD_V4_1)
+	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
+#endif /* CONFIG_NFSD_V4_1 */
 	struct list_head	recv;
 
 	struct {
@@ -197,6 +200,7 @@ struct xprt_create {
 	struct sockaddr *	srcaddr;	/* optional local address */
 	struct sockaddr *	dstaddr;	/* remote peer address */
 	size_t			addrlen;
+	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
 };
 
 struct xprt_class {
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index 836f15c..0a8f2d0 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -266,6 +266,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
 		.srcaddr = args->saddress,
 		.dstaddr = args->address,
 		.addrlen = args->addrsize,
+		.bc_sock = args->bc_sock,
 	};
 	char servername[48];
 
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 7a2a90f..90b2232 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -49,6 +49,7 @@
 #include <linux/sunrpc/msg_prot.h>
 #include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/stats.h>
+#include <linux/sunrpc/xprt.h>
 
 #define RPCDBG_FACILITY	RPCDBG_SVCXPRT
 
@@ -790,6 +791,9 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	int		len;
 	struct kvec *vec;
 	int pnum, vlen;
+#if defined(CONFIG_NFSD_V4_1)
+	struct rpc_rqst *req = NULL;
+#endif
 
 	dprintk("svc: tcp_recv %p data %d conn %d close %d\n",
 		svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags),
@@ -856,12 +860,73 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	len = svsk->sk_reclen;
 	set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags);
 
+	/*
+	 * We have enough data for the whole tcp record. Let's try and read the
+	 * first 8 bytes to get the xid and the call direction. We can use this
+	 * to figure out if this is a call or a reply to a callback. If
+	 * sk_reclen is < 8 (xid and calldir), then this is a malformed packet.
+	 * In that case, don't bother with the calldir and just read the data.
+	 * It will be rejected in svc_process.
+	 */
+
 	vec = rqstp->rq_vec;
 	vec[0] = rqstp->rq_arg.head[0];
 	vlen = PAGE_SIZE;
+
+	if (len >= 8) {
+		u32 *p;
+		u32 xid;
+		u32 calldir;
+
+		len = svc_recvfrom(rqstp, vec, 1, 8);
+		if (len < 0)
+			goto error;
+
+		p = (u32 *)rqstp->rq_arg.head[0].iov_base;
+		xid = *p++;
+		calldir = *p;
+
+#if defined(CONFIG_NFSD_V4_1)
+		if (calldir) {
+			/* REPLY */
+			if (svsk->sk_bc_xprt)
+				req = xprt_lookup_rqst(svsk->sk_bc_xprt, xid);
+			if (req) {
+				memcpy(&req->rq_private_buf, &req->rq_rcv_buf,
+					sizeof(struct xdr_buf));
+				/* copy the xid and call direction */
+				memcpy(req->rq_private_buf.head[0].iov_base,
+					rqstp->rq_arg.head[0].iov_base, 8);
+				vec[0] = req->rq_private_buf.head[0];
+			} else
+				printk(KERN_NOTICE
+					"%s: Got unrecognized reply: "
+					"calldir 0x%x sk_bc_xprt %p xid %08x\n",
+					__func__, ntohl(calldir),
+					svsk->sk_bc_xprt, xid);
+		}
+
+		if (!calldir || !req)
+			vec[0] = rqstp->rq_arg.head[0];
+
+#else /* CONFIG_NFSD_V4_1 */
+		vec[0] = rqstp->rq_arg.head[0];
+#endif /* CONFIG_NFSD_V4_1 */
+		vec[0].iov_base += 8;
+		vec[0].iov_len -= 8;
+		len = svsk->sk_reclen - 8;
+		vlen -= 8;
+	}
+
 	pnum = 1;
 	while (vlen < len) {
+#if defined(CONFIG_NFSD_V4_1)
+		vec[pnum].iov_base = (req) ?
+			page_address(req->rq_private_buf.pages[pnum - 1]) :
+			page_address(rqstp->rq_pages[pnum]);
+#else /* CONFIG_NFSD_V4_1 */
 		vec[pnum].iov_base = page_address(rqstp->rq_pages[pnum]);
+#endif /* CONFIG_NFSD_V4_1 */
 		vec[pnum].iov_len = PAGE_SIZE;
 		pnum++;
 		vlen += PAGE_SIZE;
@@ -873,6 +938,18 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	if (len < 0)
 		goto error;
 
+	/*
+	 * Account for the 8 bytes we read earlier
+	 */
+	len += 8;
+
+#if defined(CONFIG_NFSD_V4_1)
+	if (req) {
+		xprt_complete_rqst(req->rq_task, len);
+		len = 0;
+		goto out;
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 	dprintk("svc: TCP complete record (%d bytes)\n", len);
 	rqstp->rq_arg.len = len;
 	rqstp->rq_arg.page_base = 0;
@@ -886,6 +963,10 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	rqstp->rq_xprt_ctxt   = NULL;
 	rqstp->rq_prot	      = IPPROTO_TCP;
 
+#if defined(CONFIG_NFSD_V4_1)
+out:
+#endif /* CONFIG_NFSD_V4_1 */
+
 	/* Reset TCP read info */
 	svsk->sk_reclen = 0;
 	svsk->sk_tcplen = 0;
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index 62098d1..7fac90c 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -998,6 +998,27 @@ void xprt_release(struct rpc_task *task)
 	spin_unlock(&xprt->reserve_lock);
 }
 
+/*
+ * The autoclose function for the back channel
+ *
+ * The callback channel should never close the channel,
+ * let the forechannel do that.
+ */
+static void bc_autoclose(struct work_struct *work)
+{
+	return;
+}
+
+
+/*
+ * The autodisconnect routine for the back channel. We never disconnect
+ */
+static void
+bc_init_autodisconnect(unsigned long data)
+{
+	return;
+}
+
 /**
  * xprt_create_transport - create an RPC transport
  * @args: rpc transport creation arguments
@@ -1034,9 +1055,16 @@ found:
 
 	INIT_LIST_HEAD(&xprt->free);
 	INIT_LIST_HEAD(&xprt->recv);
-	INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
-	setup_timer(&xprt->timer, xprt_init_autodisconnect,
-			(unsigned long)xprt);
+	if (args->bc_sock) {
+		INIT_WORK(&xprt->task_cleanup, bc_autoclose);
+		setup_timer(&xprt->timer, bc_init_autodisconnect,
+			    (unsigned long)xprt);
+	} else {
+		INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
+		setup_timer(&xprt->timer, xprt_init_autodisconnect,
+			    (unsigned long)xprt);
+	}
+
 	xprt->last_used = jiffies;
 	xprt->cwnd = RPC_INITCWND;
 	xprt->bind_index = 0;
@@ -1056,6 +1084,13 @@ found:
 	dprintk("RPC:       created transport %p with %u slots\n", xprt,
 			xprt->max_reqs);
 
+	/*
+	 * Since we don't want connections for the backchannel, we set
+	 * the xprt status to connected
+	 */
+	if (args->bc_sock)
+		xprt_set_connected(xprt);
+
 	return xprt;
 }
 
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 29c71e6..f90c7c3 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -32,6 +32,7 @@
 #include <linux/tcp.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/sched.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/xprtsock.h>
 #include <linux/file.h>
 
@@ -1884,6 +1885,221 @@ static void xs_tcp_print_stats(struct rpc_xprt *xprt, struct seq_file *seq)
 			xprt->stat.bklog_u);
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * The connect worker for the backchannel
+ * This should never be called as we should never need to connect
+ */
+static void bc_connect_worker(struct work_struct *work)
+{
+	BUG();
+}
+
+/*
+ * The set_port routine of the rpc_xprt_ops. This is related to the portmapper
+ * and should never be called
+ */
+
+static void bc_set_port(struct rpc_xprt *xprt, unsigned short port)
+{
+	BUG();
+}
+
+/*
+ * The connect routine for the backchannel rpc_xprt ops
+ * Again, should never be called!
+ */
+
+static void bc_connect(struct rpc_task *task)
+{
+	BUG();
+}
+
+struct rpc_buffer {
+	size_t	len;
+	char	data[];
+};
+/*
+ * Allocate a bunch of pages for a scratch buffer for the rpc code. The reason
+ * we allocate pages instead doing a kmalloc like rpc_malloc is because we want
+ * to use the server side send routines.
+ */
+void *bc_malloc(struct rpc_task *task, size_t size)
+{
+	struct page *page;
+	struct rpc_buffer *buf;
+
+	BUG_ON(size > PAGE_SIZE - sizeof(struct rpc_buffer));
+	page = alloc_page(GFP_KERNEL);
+
+	if (!page)
+		return NULL;
+
+	buf = page_address(page);
+	buf->len = PAGE_SIZE;
+
+	return buf->data;
+}
+
+/*
+ * Free the space allocated in the bc_alloc routine
+ */
+void bc_free(void *buffer)
+{
+	struct rpc_buffer *buf;
+
+	if (!buffer)
+		return;
+
+	buf = container_of(buffer, struct rpc_buffer, data);
+	free_pages((unsigned long)buf, get_order(buf->len));
+}
+
+/*
+ * Use the svc_sock to send the callback. Must be called with svsk->sk_mutex
+ * held. Borrows heavily from svc_tcp_sendto and xs_tcp_semd_request.
+ */
+static int bc_sendto(struct rpc_rqst *req)
+{
+	int total_len;
+	int len;
+	int size;
+	int result;
+	struct xdr_buf *xbufp = &req->rq_snd_buf;
+	struct page **pages = xbufp->pages;
+	unsigned int flags = MSG_MORE;
+	unsigned int pglen = xbufp->page_len;
+	size_t base = xbufp->page_base;
+	struct rpc_xprt *xprt = req->rq_xprt;
+	struct sock_xprt *transport =
+				container_of(xprt, struct sock_xprt, xprt);
+	struct socket *sock = transport->sock;
+
+	total_len = xbufp->len;
+
+	/*
+	 * Set up the rpc header and record marker stuff
+	 */
+	xs_encode_tcp_record_marker(xbufp);
+
+	/*
+	 * The RPC message is divided into 3 pieces:
+	 * - The header: This is what most of the smaller RPC messages consist
+	 *   of. Often the whole message is in this.
+	 *
+	 *   - xdr->pages: This is a list of pages that contain data, for
+	 *   example in a write request or while using rpcsec gss
+	 *
+	 *   - The tail: This is the rest of the rpc message
+	 *
+	 *  First we send the header, then the pages and then finally the tail.
+	 *  The code borrows heavily from svc_sendto.
+	 */
+
+	/*
+	 * Send the head
+	 */
+	if (total_len == xbufp->head[0].iov_len)
+		flags = 0;
+
+	len = sock->ops->sendpage(sock, virt_to_page(xbufp->head[0].iov_base),
+			(unsigned long)xbufp->head[0].iov_base & ~PAGE_MASK,
+			xbufp->head[0].iov_len, flags);
+
+	if (len != xbufp->head[0].iov_len)
+		goto out;
+
+	/*
+	 * send page data
+	 *
+	 * Check the amount of data to be sent. If it is less than the
+	 * remaining page, then send it else send the current page
+	 */
+
+	size = PAGE_SIZE - base < pglen ? PAGE_SIZE - base : pglen;
+	while (pglen > 0) {
+		if (total_len == size)
+			flags = 0;
+		result = sock->ops->sendpage(sock, *pages, base, size, flags);
+		if (result > 0)
+			len += result;
+		if (result != size)
+			goto out;
+		total_len -= size;
+		pglen -= size;
+		size = PAGE_SIZE < pglen ? PAGE_SIZE : pglen;
+		base = 0;
+		pages++;
+	}
+	/*
+	 * send tail
+	 */
+	if (xbufp->tail[0].iov_len) {
+		result = sock->ops->sendpage(sock,
+			xbufp->tail[0].iov_base,
+			(unsigned long)xbufp->tail[0].iov_base & ~PAGE_MASK,
+			xbufp->tail[0].iov_len,
+			0);
+
+		if (result > 0)
+			len += result;
+	}
+out:
+	if (len != xbufp->len)
+		printk(KERN_NOTICE "Error sending entire callback!\n");
+
+	return len;
+}
+
+/*
+ * The send routine. Borrows from svc_send
+ */
+static int bc_send_request(struct rpc_task *task)
+{
+	struct rpc_rqst *req = task->tk_rqstp;
+	struct rpc_xprt *bc_xprt = req->rq_xprt;
+	struct svc_xprt	*xprt;
+	struct svc_sock         *svsk;
+	u32                     len;
+
+	dprintk("sending request with xid: %08x\n", ntohl(req->rq_xid));
+	/*
+	 * Get the server socket associated with this callback xprt
+	 */
+	svsk = bc_xprt->bc_sock;
+	xprt = &svsk->sk_xprt;
+
+	mutex_lock(&xprt->xpt_mutex);
+	if (test_bit(XPT_DEAD, &xprt->xpt_flags))
+		len = -ENOTCONN;
+	else
+		len = bc_sendto(req);
+	mutex_unlock(&xprt->xpt_mutex);
+
+	return 0;
+
+}
+
+/*
+ * The close routine. Since this is client initiated, we do nothing
+ */
+
+static void bc_close(struct rpc_xprt *xprt)
+{
+	return;
+}
+
+/*
+ * The xprt destroy routine. Again, because this connection is client
+ * initiated, we do nothing
+ */
+
+static void bc_destroy(struct rpc_xprt *xprt)
+{
+	return;
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 static struct rpc_xprt_ops xs_udp_ops = {
 	.set_buffer_size	= xs_udp_set_buffer_size,
 	.reserve_xprt		= xprt_reserve_xprt_cong,
@@ -1917,6 +2133,26 @@ static struct rpc_xprt_ops xs_tcp_ops = {
 	.print_stats		= xs_tcp_print_stats,
 };
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * The rpc_xprt_ops for the server backchannel
+ */
+
+static struct rpc_xprt_ops bc_tcp_ops = {
+	.reserve_xprt		= xprt_reserve_xprt,
+	.release_xprt		= xprt_release_xprt,
+	.set_port		= bc_set_port,
+	.connect		= bc_connect,
+	.buf_alloc		= bc_malloc,
+	.buf_free		= bc_free,
+	.send_request		= bc_send_request,
+	.set_retrans_timeout	= xprt_set_retrans_timeout_def,
+	.close			= bc_close,
+	.destroy		= bc_destroy,
+	.print_stats		= xs_tcp_print_stats,
+};
+#endif /* CONFIG_NFSD_V4_1 */
+
 static struct rpc_xprt *xs_setup_xprt(struct xprt_create *args,
 				      unsigned int slot_table_size)
 {
@@ -2049,13 +2285,31 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
 	xprt->tsh_size = sizeof(rpc_fraghdr) / sizeof(u32);
 	xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
 
-	xprt->bind_timeout = XS_BIND_TO;
-	xprt->connect_timeout = XS_TCP_CONN_TO;
-	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
-	xprt->idle_timeout = XS_IDLE_DISC_TO;
+#ifdef CONFIG_NFSD_V4_1
+	if (args->bc_sock) {
+		/* backchannel */
+		xprt_set_bound(xprt);
+		INIT_DELAYED_WORK(&transport->connect_worker,
+				  bc_connect_worker);
+		xprt->bind_timeout = 0;
+		xprt->connect_timeout = 0;
+		xprt->reestablish_timeout = 0;
+		xprt->idle_timeout = (~0);
 
-	xprt->ops = &xs_tcp_ops;
-	xprt->timeout = &xs_tcp_default_timeout;
+		/*
+		 * The backchannel uses the same socket connection as the
+		 * forechannel
+		 */
+		xprt->bc_sock = args->bc_sock;
+		xprt->bc_sock->sk_bc_xprt = xprt;
+		transport->sock = xprt->bc_sock->sk_sock;
+		transport->inet = xprt->bc_sock->sk_sk;
+
+		xprt->ops = &bc_tcp_ops;
+
+		goto next;
+	}
+#endif /* CONFIG_NFSD_V4_1 */
 
 	switch (addr->sa_family) {
 	case AF_INET:
@@ -2063,13 +2317,31 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
 			xprt_set_bound(xprt);
 
 		INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_connect_worker4);
-		xs_format_ipv4_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP);
 		break;
 	case AF_INET6:
 		if (((struct sockaddr_in6 *)addr)->sin6_port != htons(0))
 			xprt_set_bound(xprt);
 
 		INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_connect_worker6);
+		break;
+	}
+	xprt->bind_timeout = XS_BIND_TO;
+	xprt->connect_timeout = XS_TCP_CONN_TO;
+	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
+	xprt->idle_timeout = XS_IDLE_DISC_TO;
+
+	xprt->ops = &xs_tcp_ops;
+
+#ifdef CONFIG_NFSD_V4_1
+next:
+#endif /* CONFIG_NFSD_V4_1 */
+	xprt->timeout = &xs_tcp_default_timeout;
+
+	switch (addr->sa_family) {
+	case AF_INET:
+		xs_format_ipv4_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP);
+		break;
+	case AF_INET6:
 		xs_format_ipv6_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP6);
 		break;
 	default:
-- 
1.6.2.1

Re: [PATCH v2 36/47] nfsd41: sunrpc: Added rpc server-side backchannel handling

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:34:21AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> FIXME: bhalevy: write up commit message

?--b.

> 
> Signed-off-by: Mike Sager <sager@netapp.com>
> Signed-off-by: Marc Eshel <eshel@almaden.ibm.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> 
> When the call direction is a reply, copy the xid and call direction into the
> req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns
> rpc_garbage.
> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  include/linux/sunrpc/clnt.h    |    1 +
>  include/linux/sunrpc/svcsock.h |    1 +
>  include/linux/sunrpc/xprt.h    |    4 +
>  net/sunrpc/clnt.c              |    1 +
>  net/sunrpc/svcsock.c           |   81 +++++++++++
>  net/sunrpc/xprt.c              |   41 ++++++-
>  net/sunrpc/xprtsock.c          |  286 +++++++++++++++++++++++++++++++++++++++-
>  7 files changed, 405 insertions(+), 10 deletions(-)
> 
> diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
> index 928c547..0b32b00 100644
> --- a/include/linux/sunrpc/clnt.h
> +++ b/include/linux/sunrpc/clnt.h
> @@ -111,6 +111,7 @@ struct rpc_create_args {
>  	rpc_authflavor_t	authflavor;
>  	unsigned long		flags;
>  	char			*client_name;
> +	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
>  };
>  
>  /* Values for "flags" field */
> diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h
> index 483e103..e8fda10 100644
> --- a/include/linux/sunrpc/svcsock.h
> +++ b/include/linux/sunrpc/svcsock.h
> @@ -28,6 +28,7 @@ struct svc_sock {
>  	/* private TCP part */
>  	u32			sk_reclen;	/* length of record */
>  	u32			sk_tcplen;	/* current read length */
> +	struct rpc_xprt	       *sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
>  };
>  
>  /*
> diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
> index 11fc71d..e9ffee3 100644
> --- a/include/linux/sunrpc/xprt.h
> +++ b/include/linux/sunrpc/xprt.h
> @@ -174,6 +174,9 @@ struct rpc_xprt {
>  	spinlock_t		reserve_lock;	/* lock slot table */
>  	u32			xid;		/* Next XID value to use */
>  	struct rpc_task *	snd_task;	/* Task blocked in send */
> +#if defined(CONFIG_NFSD_V4_1)
> +	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
> +#endif /* CONFIG_NFSD_V4_1 */
>  	struct list_head	recv;
>  
>  	struct {
> @@ -197,6 +200,7 @@ struct xprt_create {
>  	struct sockaddr *	srcaddr;	/* optional local address */
>  	struct sockaddr *	dstaddr;	/* remote peer address */
>  	size_t			addrlen;
> +	struct svc_sock		*bc_sock;	/* NFSv4.1 backchannel */
>  };
>  
>  struct xprt_class {
> diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
> index 836f15c..0a8f2d0 100644
> --- a/net/sunrpc/clnt.c
> +++ b/net/sunrpc/clnt.c
> @@ -266,6 +266,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
>  		.srcaddr = args->saddress,
>  		.dstaddr = args->address,
>  		.addrlen = args->addrsize,
> +		.bc_sock = args->bc_sock,
>  	};
>  	char servername[48];
>  
> diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
> index 7a2a90f..90b2232 100644
> --- a/net/sunrpc/svcsock.c
> +++ b/net/sunrpc/svcsock.c
> @@ -49,6 +49,7 @@
>  #include <linux/sunrpc/msg_prot.h>
>  #include <linux/sunrpc/svcsock.h>
>  #include <linux/sunrpc/stats.h>
> +#include <linux/sunrpc/xprt.h>
>  
>  #define RPCDBG_FACILITY	RPCDBG_SVCXPRT
>  
> @@ -790,6 +791,9 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
>  	int		len;
>  	struct kvec *vec;
>  	int pnum, vlen;
> +#if defined(CONFIG_NFSD_V4_1)
> +	struct rpc_rqst *req = NULL;
> +#endif
>  
>  	dprintk("svc: tcp_recv %p data %d conn %d close %d\n",
>  		svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags),
> @@ -856,12 +860,73 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
>  	len = svsk->sk_reclen;
>  	set_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags);
>  
> +	/*
> +	 * We have enough data for the whole tcp record. Let's try and read the
> +	 * first 8 bytes to get the xid and the call direction. We can use this
> +	 * to figure out if this is a call or a reply to a callback. If
> +	 * sk_reclen is < 8 (xid and calldir), then this is a malformed packet.
> +	 * In that case, don't bother with the calldir and just read the data.
> +	 * It will be rejected in svc_process.
> +	 */
> +
>  	vec = rqstp->rq_vec;
>  	vec[0] = rqstp->rq_arg.head[0];
>  	vlen = PAGE_SIZE;
> +
> +	if (len >= 8) {
> +		u32 *p;
> +		u32 xid;
> +		u32 calldir;
> +
> +		len = svc_recvfrom(rqstp, vec, 1, 8);
> +		if (len < 0)
> +			goto error;
> +
> +		p = (u32 *)rqstp->rq_arg.head[0].iov_base;
> +		xid = *p++;
> +		calldir = *p;
> +
> +#if defined(CONFIG_NFSD_V4_1)
> +		if (calldir) {
> +			/* REPLY */
> +			if (svsk->sk_bc_xprt)
> +				req = xprt_lookup_rqst(svsk->sk_bc_xprt, xid);
> +			if (req) {
> +				memcpy(&req->rq_private_buf, &req->rq_rcv_buf,
> +					sizeof(struct xdr_buf));
> +				/* copy the xid and call direction */
> +				memcpy(req->rq_private_buf.head[0].iov_base,
> +					rqstp->rq_arg.head[0].iov_base, 8);
> +				vec[0] = req->rq_private_buf.head[0];
> +			} else
> +				printk(KERN_NOTICE
> +					"%s: Got unrecognized reply: "
> +					"calldir 0x%x sk_bc_xprt %p xid %08x\n",
> +					__func__, ntohl(calldir),
> +					svsk->sk_bc_xprt, xid);
> +		}
> +
> +		if (!calldir || !req)
> +			vec[0] = rqstp->rq_arg.head[0];
> +
> +#else /* CONFIG_NFSD_V4_1 */
> +		vec[0] = rqstp->rq_arg.head[0];
> +#endif /* CONFIG_NFSD_V4_1 */
> +		vec[0].iov_base += 8;
> +		vec[0].iov_len -= 8;
> +		len = svsk->sk_reclen - 8;
> +		vlen -= 8;
> +	}
> +
>  	pnum = 1;
>  	while (vlen < len) {
> +#if defined(CONFIG_NFSD_V4_1)
> +		vec[pnum].iov_base = (req) ?
> +			page_address(req->rq_private_buf.pages[pnum - 1]) :
> +			page_address(rqstp->rq_pages[pnum]);
> +#else /* CONFIG_NFSD_V4_1 */
>  		vec[pnum].iov_base = page_address(rqstp->rq_pages[pnum]);
> +#endif /* CONFIG_NFSD_V4_1 */
>  		vec[pnum].iov_len = PAGE_SIZE;
>  		pnum++;
>  		vlen += PAGE_SIZE;
> @@ -873,6 +938,18 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
>  	if (len < 0)
>  		goto error;
>  
> +	/*
> +	 * Account for the 8 bytes we read earlier
> +	 */
> +	len += 8;
> +
> +#if defined(CONFIG_NFSD_V4_1)
> +	if (req) {
> +		xprt_complete_rqst(req->rq_task, len);
> +		len = 0;
> +		goto out;
> +	}
> +#endif /* CONFIG_NFSD_V4_1 */
>  	dprintk("svc: TCP complete record (%d bytes)\n", len);
>  	rqstp->rq_arg.len = len;
>  	rqstp->rq_arg.page_base = 0;
> @@ -886,6 +963,10 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
>  	rqstp->rq_xprt_ctxt   = NULL;
>  	rqstp->rq_prot	      = IPPROTO_TCP;
>  
> +#if defined(CONFIG_NFSD_V4_1)
> +out:
> +#endif /* CONFIG_NFSD_V4_1 */
> +
>  	/* Reset TCP read info */
>  	svsk->sk_reclen = 0;
>  	svsk->sk_tcplen = 0;
> diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
> index 62098d1..7fac90c 100644
> --- a/net/sunrpc/xprt.c
> +++ b/net/sunrpc/xprt.c
> @@ -998,6 +998,27 @@ void xprt_release(struct rpc_task *task)
>  	spin_unlock(&xprt->reserve_lock);
>  }
>  
> +/*
> + * The autoclose function for the back channel
> + *
> + * The callback channel should never close the channel,
> + * let the forechannel do that.
> + */
> +static void bc_autoclose(struct work_struct *work)
> +{
> +	return;
> +}
> +
> +
> +/*
> + * The autodisconnect routine for the back channel. We never disconnect
> + */
> +static void
> +bc_init_autodisconnect(unsigned long data)
> +{
> +	return;
> +}
> +
>  /**
>   * xprt_create_transport - create an RPC transport
>   * @args: rpc transport creation arguments
> @@ -1034,9 +1055,16 @@ found:
>  
>  	INIT_LIST_HEAD(&xprt->free);
>  	INIT_LIST_HEAD(&xprt->recv);
> -	INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
> -	setup_timer(&xprt->timer, xprt_init_autodisconnect,
> -			(unsigned long)xprt);
> +	if (args->bc_sock) {
> +		INIT_WORK(&xprt->task_cleanup, bc_autoclose);
> +		setup_timer(&xprt->timer, bc_init_autodisconnect,
> +			    (unsigned long)xprt);
> +	} else {
> +		INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
> +		setup_timer(&xprt->timer, xprt_init_autodisconnect,
> +			    (unsigned long)xprt);
> +	}
> +
>  	xprt->last_used = jiffies;
>  	xprt->cwnd = RPC_INITCWND;
>  	xprt->bind_index = 0;
> @@ -1056,6 +1084,13 @@ found:
>  	dprintk("RPC:       created transport %p with %u slots\n", xprt,
>  			xprt->max_reqs);
>  
> +	/*
> +	 * Since we don't want connections for the backchannel, we set
> +	 * the xprt status to connected
> +	 */
> +	if (args->bc_sock)
> +		xprt_set_connected(xprt);
> +
>  	return xprt;
>  }
>  
> diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
> index 29c71e6..f90c7c3 100644
> --- a/net/sunrpc/xprtsock.c
> +++ b/net/sunrpc/xprtsock.c
> @@ -32,6 +32,7 @@
>  #include <linux/tcp.h>
>  #include <linux/sunrpc/clnt.h>
>  #include <linux/sunrpc/sched.h>
> +#include <linux/sunrpc/svcsock.h>
>  #include <linux/sunrpc/xprtsock.h>
>  #include <linux/file.h>
>  
> @@ -1884,6 +1885,221 @@ static void xs_tcp_print_stats(struct rpc_xprt *xprt, struct seq_file *seq)
>  			xprt->stat.bklog_u);
>  }
>  
> +#if defined(CONFIG_NFSD_V4_1)
> +/*
> + * The connect worker for the backchannel
> + * This should never be called as we should never need to connect
> + */
> +static void bc_connect_worker(struct work_struct *work)
> +{
> +	BUG();
> +}
> +
> +/*
> + * The set_port routine of the rpc_xprt_ops. This is related to the portmapper
> + * and should never be called
> + */
> +
> +static void bc_set_port(struct rpc_xprt *xprt, unsigned short port)
> +{
> +	BUG();
> +}
> +
> +/*
> + * The connect routine for the backchannel rpc_xprt ops
> + * Again, should never be called!
> + */
> +
> +static void bc_connect(struct rpc_task *task)
> +{
> +	BUG();
> +}
> +
> +struct rpc_buffer {
> +	size_t	len;
> +	char	data[];
> +};
> +/*
> + * Allocate a bunch of pages for a scratch buffer for the rpc code. The reason
> + * we allocate pages instead doing a kmalloc like rpc_malloc is because we want
> + * to use the server side send routines.
> + */
> +void *bc_malloc(struct rpc_task *task, size_t size)
> +{
> +	struct page *page;
> +	struct rpc_buffer *buf;
> +
> +	BUG_ON(size > PAGE_SIZE - sizeof(struct rpc_buffer));
> +	page = alloc_page(GFP_KERNEL);
> +
> +	if (!page)
> +		return NULL;
> +
> +	buf = page_address(page);
> +	buf->len = PAGE_SIZE;
> +
> +	return buf->data;
> +}
> +
> +/*
> + * Free the space allocated in the bc_alloc routine
> + */
> +void bc_free(void *buffer)
> +{
> +	struct rpc_buffer *buf;
> +
> +	if (!buffer)
> +		return;
> +
> +	buf = container_of(buffer, struct rpc_buffer, data);
> +	free_pages((unsigned long)buf, get_order(buf->len));
> +}
> +
> +/*
> + * Use the svc_sock to send the callback. Must be called with svsk->sk_mutex
> + * held. Borrows heavily from svc_tcp_sendto and xs_tcp_semd_request.
> + */
> +static int bc_sendto(struct rpc_rqst *req)
> +{
> +	int total_len;
> +	int len;
> +	int size;
> +	int result;
> +	struct xdr_buf *xbufp = &req->rq_snd_buf;
> +	struct page **pages = xbufp->pages;
> +	unsigned int flags = MSG_MORE;
> +	unsigned int pglen = xbufp->page_len;
> +	size_t base = xbufp->page_base;
> +	struct rpc_xprt *xprt = req->rq_xprt;
> +	struct sock_xprt *transport =
> +				container_of(xprt, struct sock_xprt, xprt);
> +	struct socket *sock = transport->sock;
> +
> +	total_len = xbufp->len;
> +
> +	/*
> +	 * Set up the rpc header and record marker stuff
> +	 */
> +	xs_encode_tcp_record_marker(xbufp);
> +
> +	/*
> +	 * The RPC message is divided into 3 pieces:
> +	 * - The header: This is what most of the smaller RPC messages consist
> +	 *   of. Often the whole message is in this.
> +	 *
> +	 *   - xdr->pages: This is a list of pages that contain data, for
> +	 *   example in a write request or while using rpcsec gss
> +	 *
> +	 *   - The tail: This is the rest of the rpc message
> +	 *
> +	 *  First we send the header, then the pages and then finally the tail.
> +	 *  The code borrows heavily from svc_sendto.
> +	 */
> +
> +	/*
> +	 * Send the head
> +	 */
> +	if (total_len == xbufp->head[0].iov_len)
> +		flags = 0;
> +
> +	len = sock->ops->sendpage(sock, virt_to_page(xbufp->head[0].iov_base),
> +			(unsigned long)xbufp->head[0].iov_base & ~PAGE_MASK,
> +			xbufp->head[0].iov_len, flags);
> +
> +	if (len != xbufp->head[0].iov_len)
> +		goto out;
> +
> +	/*
> +	 * send page data
> +	 *
> +	 * Check the amount of data to be sent. If it is less than the
> +	 * remaining page, then send it else send the current page
> +	 */
> +
> +	size = PAGE_SIZE - base < pglen ? PAGE_SIZE - base : pglen;
> +	while (pglen > 0) {
> +		if (total_len == size)
> +			flags = 0;
> +		result = sock->ops->sendpage(sock, *pages, base, size, flags);
> +		if (result > 0)
> +			len += result;
> +		if (result != size)
> +			goto out;
> +		total_len -= size;
> +		pglen -= size;
> +		size = PAGE_SIZE < pglen ? PAGE_SIZE : pglen;
> +		base = 0;
> +		pages++;
> +	}
> +	/*
> +	 * send tail
> +	 */
> +	if (xbufp->tail[0].iov_len) {
> +		result = sock->ops->sendpage(sock,
> +			xbufp->tail[0].iov_base,
> +			(unsigned long)xbufp->tail[0].iov_base & ~PAGE_MASK,
> +			xbufp->tail[0].iov_len,
> +			0);
> +
> +		if (result > 0)
> +			len += result;
> +	}
> +out:
> +	if (len != xbufp->len)
> +		printk(KERN_NOTICE "Error sending entire callback!\n");
> +
> +	return len;
> +}
> +
> +/*
> + * The send routine. Borrows from svc_send
> + */
> +static int bc_send_request(struct rpc_task *task)
> +{
> +	struct rpc_rqst *req = task->tk_rqstp;
> +	struct rpc_xprt *bc_xprt = req->rq_xprt;
> +	struct svc_xprt	*xprt;
> +	struct svc_sock         *svsk;
> +	u32                     len;
> +
> +	dprintk("sending request with xid: %08x\n", ntohl(req->rq_xid));
> +	/*
> +	 * Get the server socket associated with this callback xprt
> +	 */
> +	svsk = bc_xprt->bc_sock;
> +	xprt = &svsk->sk_xprt;
> +
> +	mutex_lock(&xprt->xpt_mutex);
> +	if (test_bit(XPT_DEAD, &xprt->xpt_flags))
> +		len = -ENOTCONN;
> +	else
> +		len = bc_sendto(req);
> +	mutex_unlock(&xprt->xpt_mutex);
> +
> +	return 0;
> +
> +}
> +
> +/*
> + * The close routine. Since this is client initiated, we do nothing
> + */
> +
> +static void bc_close(struct rpc_xprt *xprt)
> +{
> +	return;
> +}
> +
> +/*
> + * The xprt destroy routine. Again, because this connection is client
> + * initiated, we do nothing
> + */
> +
> +static void bc_destroy(struct rpc_xprt *xprt)
> +{
> +	return;
> +}
> +#endif /* CONFIG_NFSD_V4_1 */
> +
>  static struct rpc_xprt_ops xs_udp_ops = {
>  	.set_buffer_size	= xs_udp_set_buffer_size,
>  	.reserve_xprt		= xprt_reserve_xprt_cong,
> @@ -1917,6 +2133,26 @@ static struct rpc_xprt_ops xs_tcp_ops = {
>  	.print_stats		= xs_tcp_print_stats,
>  };
>  
> +#if defined(CONFIG_NFSD_V4_1)
> +/*
> + * The rpc_xprt_ops for the server backchannel
> + */
> +
> +static struct rpc_xprt_ops bc_tcp_ops = {
> +	.reserve_xprt		= xprt_reserve_xprt,
> +	.release_xprt		= xprt_release_xprt,
> +	.set_port		= bc_set_port,
> +	.connect		= bc_connect,
> +	.buf_alloc		= bc_malloc,
> +	.buf_free		= bc_free,
> +	.send_request		= bc_send_request,
> +	.set_retrans_timeout	= xprt_set_retrans_timeout_def,
> +	.close			= bc_close,
> +	.destroy		= bc_destroy,
> +	.print_stats		= xs_tcp_print_stats,
> +};
> +#endif /* CONFIG_NFSD_V4_1 */
> +
>  static struct rpc_xprt *xs_setup_xprt(struct xprt_create *args,
>  				      unsigned int slot_table_size)
>  {
> @@ -2049,13 +2285,31 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
>  	xprt->tsh_size = sizeof(rpc_fraghdr) / sizeof(u32);
>  	xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
>  
> -	xprt->bind_timeout = XS_BIND_TO;
> -	xprt->connect_timeout = XS_TCP_CONN_TO;
> -	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
> -	xprt->idle_timeout = XS_IDLE_DISC_TO;
> +#ifdef CONFIG_NFSD_V4_1
> +	if (args->bc_sock) {
> +		/* backchannel */
> +		xprt_set_bound(xprt);
> +		INIT_DELAYED_WORK(&transport->connect_worker,
> +				  bc_connect_worker);
> +		xprt->bind_timeout = 0;
> +		xprt->connect_timeout = 0;
> +		xprt->reestablish_timeout = 0;
> +		xprt->idle_timeout = (~0);
>  
> -	xprt->ops = &xs_tcp_ops;
> -	xprt->timeout = &xs_tcp_default_timeout;
> +		/*
> +		 * The backchannel uses the same socket connection as the
> +		 * forechannel
> +		 */
> +		xprt->bc_sock = args->bc_sock;
> +		xprt->bc_sock->sk_bc_xprt = xprt;
> +		transport->sock = xprt->bc_sock->sk_sock;
> +		transport->inet = xprt->bc_sock->sk_sk;
> +
> +		xprt->ops = &bc_tcp_ops;
> +
> +		goto next;
> +	}
> +#endif /* CONFIG_NFSD_V4_1 */
>  
>  	switch (addr->sa_family) {
>  	case AF_INET:
> @@ -2063,13 +2317,31 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
>  			xprt_set_bound(xprt);
>  
>  		INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_connect_worker4);
> -		xs_format_ipv4_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP);
>  		break;
>  	case AF_INET6:
>  		if (((struct sockaddr_in6 *)addr)->sin6_port != htons(0))
>  			xprt_set_bound(xprt);
>  
>  		INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_connect_worker6);
> +		break;
> +	}
> +	xprt->bind_timeout = XS_BIND_TO;
> +	xprt->connect_timeout = XS_TCP_CONN_TO;
> +	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
> +	xprt->idle_timeout = XS_IDLE_DISC_TO;
> +
> +	xprt->ops = &xs_tcp_ops;
> +
> +#ifdef CONFIG_NFSD_V4_1
> +next:
> +#endif /* CONFIG_NFSD_V4_1 */
> +	xprt->timeout = &xs_tcp_default_timeout;
> +
> +	switch (addr->sa_family) {
> +	case AF_INET:
> +		xs_format_ipv4_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP);
> +		break;
> +	case AF_INET6:
>  		xs_format_ipv6_peer_addresses(xprt, "tcp", RPCBIND_NETID_TCP6);
>  		break;
>  	default:
> -- 
> 1.6.2.1
> 

[PATCH v2 37/47] nfsd41: callback infrastructure

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Keep the xprt used for create_session in cl_cb_xprt.
Mark cl_callback.cb_minorversion = 1 and remember
the client provided cl_callback.cb_prog rpc program number.
Use it to probe the callback path.

Define xdr sizes and code nfs4_cb_compound header to be able
to send a null callback rpc.

Signed-off-by: Andy Adamson<andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   35 ++++++++++++++++++++++++++++++++---
 fs/nfsd/nfs4state.c    |   10 ++++++++++
 2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index d8f2631..6b7ef87 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
-        NFSPROC4_CLNT_CB_NULL = 0,
+	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -135,13 +146,19 @@ xdr_error:                                      \
 		return -EIO; \
 	} \
 } while (0)
+#define COPYMEM(x, nbytes) do {                \
+	memcpy((x), p, nbytes);                \
+	p += XDR_QUADLEN(nbytes);              \
+} while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -405,6 +422,17 @@ static struct rpc_clnt *setup_callback_client(struct nfs4_client *clp)
 	addr.sin_family = AF_INET;
 	addr.sin_port = htons(cb->cb_port);
 	addr.sin_addr.s_addr = htonl(cb->cb_addr);
+#if defined(CONFIG_NFSD_V4_1)
+	if (cb->cb_minorversion) {
+		BUG_ON(cb->cb_minorversion != 1);
+		args.bc_sock = container_of(clp->cl_cb_xprt, struct svc_sock,
+					    sk_xprt);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
+
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
 
 	/* Create RPC client */
 	client = rpc_create(&args);
@@ -445,6 +473,7 @@ static int do_probe_callback(void *data)
 	put_nfs4_client(clp);
 	return 0;
 out_release_client:
+	dprintk("NFSD: synchronous CB_NULL failed. status=%d\n", status);
 	rpc_shutdown_client(client);
 out_err:
 	dprintk("NFSD: warning: no callback path to client %.*s: error %d\n",
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 189f1c2..ef1e372 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -663,6 +663,8 @@ free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
 #if defined(CONFIG_NFSD_V4_1)
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	nfsd4_release_respages(clp->cl_slot.sl_cache_entry.ce_respages,
 			     clp->cl_slot.sl_cache_entry.ce_resused);
 #endif /* CONFIG_NFSD_V4_1 */
@@ -1411,6 +1413,14 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			unconf->cl_callback.cb_minorversion = 1;
+			unconf->cl_cb_seq_nr = 0;
+			unconf->cl_callback.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
-- 
1.6.2.1

[PATCH v2 38/47] nfsd41: Remember the auth flavor to use for callbacks

[Benny Halevy]

From: Ricardo Labiaga <Ricardo.Labiaga@netapp.com>

The callbacks will be sent using the same authentication flavor that
was used during session creation.  We'll add code to remember the
principal in the case RPCSEC_GSS in a separate patch.

Signed-off-by: Ricardo Labiaga <Ricardo.Labiaga@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index ef1e372..eb3d0dc 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1307,6 +1307,7 @@ out_new:
 	copy_verf(new, &verf);
 	copy_cred(&new->cl_cred, &rqstp->rq_cred);
 	new->cl_addr = ip_addr;
+	new->cl_flavor = rqstp->rq_flavor;
 	gen_clid(new);
 	gen_confirm(new);
 	add_to_unconfirmed(new, strhashval);
-- 
1.6.2.1

[PATCH v2 39/47] nfsd41: introduce cl_cb_mutex

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

To be used for serializing nfs41 callbacks.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4state.c        |    1 +
 include/linux/nfsd/state.h |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index eb3d0dc..e5c7ddb 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -741,6 +741,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
 	INIT_LIST_HEAD(&clp->cl_delegations);
 #if defined(CONFIG_NFSD_V4_1)
 	INIT_LIST_HEAD(&clp->cl_sessions);
+	mutex_init(&clp->cl_cb_mutex);
 #endif /* CONFIG_NFSD_V4_1 */
 	INIT_LIST_HEAD(&clp->cl_lru);
 	return clp;
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index b78021a..f2783d4 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -197,6 +197,7 @@ struct nfs4_client {
 	struct nfs4_sessionid	cl_sessionid;
 
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
+	struct mutex		cl_cb_mutex;
 #endif /* CONFIG_NFSD_V4_1 */
 };
 
-- 
1.6.2.1

[PATCH v2 40/47] nfsd41: cb_sequence callback

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
Signed-off-by: Andy Adamson <andros@netapp.com>
[fixed indentation]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfsd/state.h |    7 ++-
 2 files changed, 124 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 6b7ef87..7ada6b1 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
 	hdr->nops++;
 }
 
+static void
+encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
+		   struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	if (hdr->minorversion == 0)
+		return;
+
+	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
+
+	WRITE32(OP_CB_SEQUENCE);
+#ifdef CONFIG_NFSD_V4_1
+	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(args->cbs_clp->cl_cb_seq_nr);
+#endif /* CONFIG_NFSD_V4_1 */
+	WRITE32(0);		/* slotid, always 0 */
+	WRITE32(0);		/* highest slotid always 0 */
+	WRITE32(0);		/* cachethis always 0 */
+	WRITE32(0); /* FIXME: support referring_call_lists */
+	hdr->nops++;
+}
+
 static int
 nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
 	return 0;
 }
 
+/*
+ * Our current back channel implmentation supports a single backchannel
+ * with a single slot.
+ */
+static int
+decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
+		   struct rpc_rqst *rqstp)
+{
+	struct nfs4_callback *cb =
+		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
+	struct nfs4_sessionid id;
+	int status;
+	u32 dummy;
+	__be32 *p;
+
+	if (cb->cb_minorversion == 0)
+		return 0;
+
+	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
+	if (status)
+		return status;
+
+	/*
+	 * If the server returns different values for sessionID, slotID or
+	 * sequence number, the server is looney tunes.
+	 */
+	status = -ESERVERFAULT;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
+#ifdef CONFIG_NFSD_V4_1
+	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
+		   NFS4_MAX_SESSIONID_LEN)) {
+		dprintk("%s Invalid session id\n", __func__);
+		goto out;
+	}
+	READ32(dummy);
+	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
+		dprintk("%s Invalid sequence number\n", __func__);
+		goto out;
+	}
+#endif /* CONFIG_NFSD_V4_1 */
+	READ32(dummy); 	/* slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid slotid\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* highest slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid highest slotid\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* target highest slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid target highest slotid\n", __func__);
+		goto out;
+	}
+	status = 0;
+out:
+	return status;
+}
+
+
 static int
 nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
 	return;
 }
 
+#if defined(CONFIG_NFSD_V4_1)
+/*
+ * FIXME: cb_sequence should support referring call lists, cachethis, and
+ * multiple slots
+ */
+static int
+nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
+{
+	u32 *ptr = (u32 *)clp->cl_sessionid.data;
+
+	dprintk("%s: %u:%u:%u:%u\n", __func__,
+		ptr[0], ptr[1], ptr[2], ptr[3]);
+
+	mutex_lock(&clp->cl_cb_mutex);
+	args->cbs_clp = clp;
+	clp->cl_cb_seq_nr++;
+	return 0;
+}
+
+static void
+nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
+{
+	u32 *ptr = (u32 *)clp->cl_sessionid.data;
+
+	dprintk("%s: %u:%u:%u:%u\n", __func__,
+		ptr[0], ptr[1], ptr[2], ptr[3]);
+
+	/* FIXME: support multiple callback slots */
+	mutex_unlock(&clp->cl_cb_mutex);
+}
+#endif /* CONFIG_NFSD_V4_1 */
+
 /*
  * called with dp->dl_count inc'ed.
  */
diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
index f2783d4..0ea8c89 100644
--- a/include/linux/nfsd/state.h
+++ b/include/linux/nfsd/state.h
@@ -61,6 +61,10 @@ typedef struct {
 #define si_stateownerid   si_opaque.so_stateownerid
 #define si_fileid         si_opaque.so_fileid
 
+struct nfsd4_cb_sequence {
+	/* args/res */
+	struct nfs4_client	*cbs_clp;
+};
 
 struct nfs4_cb_recall {
 	u32			cbr_ident;
@@ -195,7 +199,8 @@ struct nfs4_client {
 	struct nfsd4_slot	cl_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
-
+	/* We currently support a single back channel with a single slot */
+	u32			cl_cb_seq_nr;
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 	struct mutex		cl_cb_mutex;
 #endif /* CONFIG_NFSD_V4_1 */
-- 
1.6.2.1

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1
> 
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> [Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
> Signed-off-by: Andy Adamson <andros@netapp.com>
> [fixed indentation]
> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> ---
>  fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/nfsd/state.h |    7 ++-
>  2 files changed, 124 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
> index 6b7ef87..7ada6b1 100644
> --- a/fs/nfsd/nfs4callback.c
> +++ b/fs/nfsd/nfs4callback.c
> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
>  	hdr->nops++;
>  }
>  
> +static void
> +encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
> +		   struct nfs4_cb_compound_hdr *hdr)
> +{
> +	__be32 *p;
> +
> +	if (hdr->minorversion == 0)
> +		return;
> +
> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
> +
> +	WRITE32(OP_CB_SEQUENCE);
> +#ifdef CONFIG_NFSD_V4_1
> +	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
> +#endif /* CONFIG_NFSD_V4_1 */

This whole function should be under CONFIG_NFSD_V4_1.

> +	WRITE32(0);		/* slotid, always 0 */
> +	WRITE32(0);		/* highest slotid always 0 */
> +	WRITE32(0);		/* cachethis always 0 */
> +	WRITE32(0); /* FIXME: support referring_call_lists */
> +	hdr->nops++;
> +}
> +
>  static int
>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
>  {
> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
>  	return 0;
>  }
>  
> +/*
> + * Our current back channel implmentation supports a single backchannel
> + * with a single slot.
> + */
> +static int
> +decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
> +		   struct rpc_rqst *rqstp)
> +{
> +	struct nfs4_callback *cb =
> +		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
> +	struct nfs4_sessionid id;
> +	int status;
> +	u32 dummy;
> +	__be32 *p;
> +
> +	if (cb->cb_minorversion == 0)
> +		return 0;
> +
> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
> +	if (status)
> +		return status;
> +
> +	/*
> +	 * If the server returns different values for sessionID, slotID or
> +	 * sequence number, the server is looney tunes.
> +	 */
> +	status = -ESERVERFAULT;
> +
> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
> +#ifdef CONFIG_NFSD_V4_1
> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
> +		   NFS4_MAX_SESSIONID_LEN)) {
> +		dprintk("%s Invalid session id\n", __func__);
> +		goto out;
> +	}
> +	READ32(dummy);
> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
> +		dprintk("%s Invalid sequence number\n", __func__);
> +		goto out;
> +	}
> +#endif /* CONFIG_NFSD_V4_1 */

Ditto.

> +	READ32(dummy); 	/* slotid must be 0 */
> +	if (dummy != 0) {
> +		dprintk("%s Invalid slotid\n", __func__);
> +		goto out;
> +	}
> +	READ32(dummy); 	/* highest slotid must be 0 */
> +	if (dummy != 0) {
> +		dprintk("%s Invalid highest slotid\n", __func__);
> +		goto out;
> +	}
> +	READ32(dummy); 	/* target highest slotid must be 0 */
> +	if (dummy != 0) {
> +		dprintk("%s Invalid target highest slotid\n", __func__);
> +		goto out;
> +	}
> +	status = 0;
> +out:
> +	return status;
> +}
> +
> +
>  static int
>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
>  {
> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
>  	return;
>  }
>  
> +#if defined(CONFIG_NFSD_V4_1)
> +/*
> + * FIXME: cb_sequence should support referring call lists, cachethis, and
> + * multiple slots
> + */
> +static int
> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
> +{
> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> +
> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> +
> +	mutex_lock(&clp->cl_cb_mutex);

We shouldn't be holding a mutex across a callback.  Why is this needed?

--b.

> +	args->cbs_clp = clp;
> +	clp->cl_cb_seq_nr++;
> +	return 0;
> +}
> +
> +static void
> +nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
> +{
> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> +
> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> +
> +	/* FIXME: support multiple callback slots */
> +	mutex_unlock(&clp->cl_cb_mutex);
> +}
> +#endif /* CONFIG_NFSD_V4_1 */
> +
>  /*
>   * called with dp->dl_count inc'ed.
>   */
> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> index f2783d4..0ea8c89 100644
> --- a/include/linux/nfsd/state.h
> +++ b/include/linux/nfsd/state.h
> @@ -61,6 +61,10 @@ typedef struct {
>  #define si_stateownerid   si_opaque.so_stateownerid
>  #define si_fileid         si_opaque.so_fileid
>  
> +struct nfsd4_cb_sequence {
> +	/* args/res */
> +	struct nfs4_client	*cbs_clp;
> +};
>  
>  struct nfs4_cb_recall {
>  	u32			cbr_ident;
> @@ -195,7 +199,8 @@ struct nfs4_client {
>  	struct nfsd4_slot	cl_slot;	/* create_session slot */
>  	u32			cl_exchange_flags;
>  	struct nfs4_sessionid	cl_sessionid;
> -
> +	/* We currently support a single back channel with a single slot */
> +	u32			cl_cb_seq_nr;
>  	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
>  	struct mutex		cl_cb_mutex;
>  #endif /* CONFIG_NFSD_V4_1 */
> -- 
> 1.6.2.1
> 

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Benny Halevy]

On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> [fixed indentation]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
>>  include/linux/nfsd/state.h |    7 ++-
>>  2 files changed, 124 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
>> index 6b7ef87..7ada6b1 100644
>> --- a/fs/nfsd/nfs4callback.c
>> +++ b/fs/nfsd/nfs4callback.c
>> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
>>  	hdr->nops++;
>>  }
>>  
>> +static void
>> +encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
>> +		   struct nfs4_cb_compound_hdr *hdr)
>> +{
>> +	__be32 *p;
>> +
>> +	if (hdr->minorversion == 0)
>> +		return;
>> +
>> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
>> +
>> +	WRITE32(OP_CB_SEQUENCE);
>> +#ifdef CONFIG_NFSD_V4_1
>> +	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
>> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
>> +#endif /* CONFIG_NFSD_V4_1 */
> 
> This whole function should be under CONFIG_NFSD_V4_1.

Right.  I wonder how it got to this state.
Will fix.

Benny

> 
>> +	WRITE32(0);		/* slotid, always 0 */
>> +	WRITE32(0);		/* highest slotid always 0 */
>> +	WRITE32(0);		/* cachethis always 0 */
>> +	WRITE32(0); /* FIXME: support referring_call_lists */
>> +	hdr->nops++;
>> +}
>> +
>>  static int
>>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
>>  {
>> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
>>  	return 0;
>>  }
>>  
>> +/*
>> + * Our current back channel implmentation supports a single backchannel
>> + * with a single slot.
>> + */
>> +static int
>> +decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
>> +		   struct rpc_rqst *rqstp)
>> +{
>> +	struct nfs4_callback *cb =
>> +		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
>> +	struct nfs4_sessionid id;
>> +	int status;
>> +	u32 dummy;
>> +	__be32 *p;
>> +
>> +	if (cb->cb_minorversion == 0)
>> +		return 0;
>> +
>> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
>> +	if (status)
>> +		return status;
>> +
>> +	/*
>> +	 * If the server returns different values for sessionID, slotID or
>> +	 * sequence number, the server is looney tunes.
>> +	 */
>> +	status = -ESERVERFAULT;
>> +
>> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
>> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
>> +#ifdef CONFIG_NFSD_V4_1
>> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
>> +		   NFS4_MAX_SESSIONID_LEN)) {
>> +		dprintk("%s Invalid session id\n", __func__);
>> +		goto out;
>> +	}
>> +	READ32(dummy);
>> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
>> +		dprintk("%s Invalid sequence number\n", __func__);
>> +		goto out;
>> +	}
>> +#endif /* CONFIG_NFSD_V4_1 */
> 
> Ditto.
> 
>> +	READ32(dummy); 	/* slotid must be 0 */
>> +	if (dummy != 0) {
>> +		dprintk("%s Invalid slotid\n", __func__);
>> +		goto out;
>> +	}
>> +	READ32(dummy); 	/* highest slotid must be 0 */
>> +	if (dummy != 0) {
>> +		dprintk("%s Invalid highest slotid\n", __func__);
>> +		goto out;
>> +	}
>> +	READ32(dummy); 	/* target highest slotid must be 0 */
>> +	if (dummy != 0) {
>> +		dprintk("%s Invalid target highest slotid\n", __func__);
>> +		goto out;
>> +	}
>> +	status = 0;
>> +out:
>> +	return status;
>> +}
>> +
>> +
>>  static int
>>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
>>  {
>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
>>  	return;
>>  }
>>  
>> +#if defined(CONFIG_NFSD_V4_1)
>> +/*
>> + * FIXME: cb_sequence should support referring call lists, cachethis, and
>> + * multiple slots
>> + */
>> +static int
>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
>> +{
>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
>> +
>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
>> +
>> +	mutex_lock(&clp->cl_cb_mutex);
> 
> We shouldn't be holding a mutex across a callback.  Why is this needed?
> 
> --b.
> 
>> +	args->cbs_clp = clp;
>> +	clp->cl_cb_seq_nr++;
>> +	return 0;
>> +}
>> +
>> +static void
>> +nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
>> +{
>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
>> +
>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
>> +
>> +	/* FIXME: support multiple callback slots */
>> +	mutex_unlock(&clp->cl_cb_mutex);
>> +}
>> +#endif /* CONFIG_NFSD_V4_1 */
>> +
>>  /*
>>   * called with dp->dl_count inc'ed.
>>   */
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index f2783d4..0ea8c89 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -61,6 +61,10 @@ typedef struct {
>>  #define si_stateownerid   si_opaque.so_stateownerid
>>  #define si_fileid         si_opaque.so_fileid
>>  
>> +struct nfsd4_cb_sequence {
>> +	/* args/res */
>> +	struct nfs4_client	*cbs_clp;
>> +};
>>  
>>  struct nfs4_cb_recall {
>>  	u32			cbr_ident;
>> @@ -195,7 +199,8 @@ struct nfs4_client {
>>  	struct nfsd4_slot	cl_slot;	/* create_session slot */
>>  	u32			cl_exchange_flags;
>>  	struct nfs4_sessionid	cl_sessionid;
>> -
>> +	/* We currently support a single back channel with a single slot */
>> +	u32			cl_cb_seq_nr;
>>  	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
>>  	struct mutex		cl_cb_mutex;
>>  #endif /* CONFIG_NFSD_V4_1 */
>> -- 
>> 1.6.2.1
>>

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[J. Bruce Fields]

On Wed, Apr 01, 2009 at 11:43:41AM +0300, Benny Halevy wrote:
> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
> >> From: Andy Adamson <andros@netapp.com>
> >>
> >> Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1
> >>
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> [Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
> >> Signed-off-by: Andy Adamson <andros@netapp.com>
> >> [fixed indentation]
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> ---
> >>  fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
> >>  include/linux/nfsd/state.h |    7 ++-
> >>  2 files changed, 124 insertions(+), 1 deletions(-)
> >>
> >> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
> >> index 6b7ef87..7ada6b1 100644
> >> --- a/fs/nfsd/nfs4callback.c
> >> +++ b/fs/nfsd/nfs4callback.c
> >> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
> >>  	hdr->nops++;
> >>  }
> >>  
> >> +static void
> >> +encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
> >> +		   struct nfs4_cb_compound_hdr *hdr)
> >> +{
> >> +	__be32 *p;
> >> +
> >> +	if (hdr->minorversion == 0)
> >> +		return;
> >> +
> >> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
> >> +
> >> +	WRITE32(OP_CB_SEQUENCE);
> >> +#ifdef CONFIG_NFSD_V4_1
> >> +	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
> >> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
> >> +#endif /* CONFIG_NFSD_V4_1 */
> > 
> > This whole function should be under CONFIG_NFSD_V4_1.
> 
> Right.  I wonder how it got to this state.
> Will fix.

Thanks.  I'm more curious about the mutex question.

But I'm inclined to defer the callback changes to 2.6.31, so that's not
urgent.

--b.

> 
> Benny
> 
> > 
> >> +	WRITE32(0);		/* slotid, always 0 */
> >> +	WRITE32(0);		/* highest slotid always 0 */
> >> +	WRITE32(0);		/* cachethis always 0 */
> >> +	WRITE32(0); /* FIXME: support referring_call_lists */
> >> +	hdr->nops++;
> >> +}
> >> +
> >>  static int
> >>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
> >>  {
> >> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
> >>  	return 0;
> >>  }
> >>  
> >> +/*
> >> + * Our current back channel implmentation supports a single backchannel
> >> + * with a single slot.
> >> + */
> >> +static int
> >> +decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
> >> +		   struct rpc_rqst *rqstp)
> >> +{
> >> +	struct nfs4_callback *cb =
> >> +		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
> >> +	struct nfs4_sessionid id;
> >> +	int status;
> >> +	u32 dummy;
> >> +	__be32 *p;
> >> +
> >> +	if (cb->cb_minorversion == 0)
> >> +		return 0;
> >> +
> >> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
> >> +	if (status)
> >> +		return status;
> >> +
> >> +	/*
> >> +	 * If the server returns different values for sessionID, slotID or
> >> +	 * sequence number, the server is looney tunes.
> >> +	 */
> >> +	status = -ESERVERFAULT;
> >> +
> >> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
> >> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
> >> +#ifdef CONFIG_NFSD_V4_1
> >> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
> >> +		   NFS4_MAX_SESSIONID_LEN)) {
> >> +		dprintk("%s Invalid session id\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	READ32(dummy);
> >> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
> >> +		dprintk("%s Invalid sequence number\n", __func__);
> >> +		goto out;
> >> +	}
> >> +#endif /* CONFIG_NFSD_V4_1 */
> > 
> > Ditto.
> > 
> >> +	READ32(dummy); 	/* slotid must be 0 */
> >> +	if (dummy != 0) {
> >> +		dprintk("%s Invalid slotid\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	READ32(dummy); 	/* highest slotid must be 0 */
> >> +	if (dummy != 0) {
> >> +		dprintk("%s Invalid highest slotid\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	READ32(dummy); 	/* target highest slotid must be 0 */
> >> +	if (dummy != 0) {
> >> +		dprintk("%s Invalid target highest slotid\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	status = 0;
> >> +out:
> >> +	return status;
> >> +}
> >> +
> >> +
> >>  static int
> >>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
> >>  {
> >> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
> >>  	return;
> >>  }
> >>  
> >> +#if defined(CONFIG_NFSD_V4_1)
> >> +/*
> >> + * FIXME: cb_sequence should support referring call lists, cachethis, and
> >> + * multiple slots
> >> + */
> >> +static int
> >> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
> >> +{
> >> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> >> +
> >> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> >> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> >> +
> >> +	mutex_lock(&clp->cl_cb_mutex);
> > 
> > We shouldn't be holding a mutex across a callback.  Why is this needed?
> > 
> > --b.
> > 
> >> +	args->cbs_clp = clp;
> >> +	clp->cl_cb_seq_nr++;
> >> +	return 0;
> >> +}
> >> +
> >> +static void
> >> +nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
> >> +{
> >> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> >> +
> >> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> >> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> >> +
> >> +	/* FIXME: support multiple callback slots */
> >> +	mutex_unlock(&clp->cl_cb_mutex);
> >> +}
> >> +#endif /* CONFIG_NFSD_V4_1 */
> >> +
> >>  /*
> >>   * called with dp->dl_count inc'ed.
> >>   */
> >> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
> >> index f2783d4..0ea8c89 100644
> >> --- a/include/linux/nfsd/state.h
> >> +++ b/include/linux/nfsd/state.h
> >> @@ -61,6 +61,10 @@ typedef struct {
> >>  #define si_stateownerid   si_opaque.so_stateownerid
> >>  #define si_fileid         si_opaque.so_fileid
> >>  
> >> +struct nfsd4_cb_sequence {
> >> +	/* args/res */
> >> +	struct nfs4_client	*cbs_clp;
> >> +};
> >>  
> >>  struct nfs4_cb_recall {
> >>  	u32			cbr_ident;
> >> @@ -195,7 +199,8 @@ struct nfs4_client {
> >>  	struct nfsd4_slot	cl_slot;	/* create_session slot */
> >>  	u32			cl_exchange_flags;
> >>  	struct nfs4_sessionid	cl_sessionid;
> >> -
> >> +	/* We currently support a single back channel with a single slot */
> >> +	u32			cl_cb_seq_nr;
> >>  	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
> >>  	struct mutex		cl_cb_mutex;
> >>  #endif /* CONFIG_NFSD_V4_1 */
> >> -- 
> >> 1.6.2.1
> >>

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Benny Halevy]

On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1
>>
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> [Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> [fixed indentation]
>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>> ---
>>  fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
>>  include/linux/nfsd/state.h |    7 ++-
>>  2 files changed, 124 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
>> index 6b7ef87..7ada6b1 100644
>> --- a/fs/nfsd/nfs4callback.c
>> +++ b/fs/nfsd/nfs4callback.c
>> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
>>  	hdr->nops++;
>>  }
>>  
>> +static void
>> +encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
>> +		   struct nfs4_cb_compound_hdr *hdr)
>> +{
>> +	__be32 *p;
>> +
>> +	if (hdr->minorversion == 0)
>> +		return;
>> +
>> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
>> +
>> +	WRITE32(OP_CB_SEQUENCE);
>> +#ifdef CONFIG_NFSD_V4_1
>> +	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
>> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
>> +#endif /* CONFIG_NFSD_V4_1 */
> 
> This whole function should be under CONFIG_NFSD_V4_1.
> 
>> +	WRITE32(0);		/* slotid, always 0 */
>> +	WRITE32(0);		/* highest slotid always 0 */
>> +	WRITE32(0);		/* cachethis always 0 */
>> +	WRITE32(0); /* FIXME: support referring_call_lists */
>> +	hdr->nops++;
>> +}
>> +
>>  static int
>>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
>>  {
>> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
>>  	return 0;
>>  }
>>  
>> +/*
>> + * Our current back channel implmentation supports a single backchannel
>> + * with a single slot.
>> + */
>> +static int
>> +decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
>> +		   struct rpc_rqst *rqstp)
>> +{
>> +	struct nfs4_callback *cb =
>> +		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
>> +	struct nfs4_sessionid id;
>> +	int status;
>> +	u32 dummy;
>> +	__be32 *p;
>> +
>> +	if (cb->cb_minorversion == 0)
>> +		return 0;
>> +
>> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
>> +	if (status)
>> +		return status;
>> +
>> +	/*
>> +	 * If the server returns different values for sessionID, slotID or
>> +	 * sequence number, the server is looney tunes.
>> +	 */
>> +	status = -ESERVERFAULT;
>> +
>> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
>> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
>> +#ifdef CONFIG_NFSD_V4_1
>> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
>> +		   NFS4_MAX_SESSIONID_LEN)) {
>> +		dprintk("%s Invalid session id\n", __func__);
>> +		goto out;
>> +	}
>> +	READ32(dummy);
>> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
>> +		dprintk("%s Invalid sequence number\n", __func__);
>> +		goto out;
>> +	}
>> +#endif /* CONFIG_NFSD_V4_1 */
> 
> Ditto.
> 
>> +	READ32(dummy); 	/* slotid must be 0 */
>> +	if (dummy != 0) {
>> +		dprintk("%s Invalid slotid\n", __func__);
>> +		goto out;
>> +	}
>> +	READ32(dummy); 	/* highest slotid must be 0 */
>> +	if (dummy != 0) {
>> +		dprintk("%s Invalid highest slotid\n", __func__);
>> +		goto out;
>> +	}
>> +	READ32(dummy); 	/* target highest slotid must be 0 */
>> +	if (dummy != 0) {
>> +		dprintk("%s Invalid target highest slotid\n", __func__);
>> +		goto out;
>> +	}
>> +	status = 0;
>> +out:
>> +	return status;
>> +}
>> +
>> +
>>  static int
>>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
>>  {
>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
>>  	return;
>>  }
>>  
>> +#if defined(CONFIG_NFSD_V4_1)
>> +/*
>> + * FIXME: cb_sequence should support referring call lists, cachethis, and
>> + * multiple slots
>> + */
>> +static int
>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
>> +{
>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
>> +
>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
>> +
>> +	mutex_lock(&clp->cl_cb_mutex);
> 
> We shouldn't be holding a mutex across a callback.  Why is this needed?

Just a simple way to limit concurrency to 1 and not deal with
multiple slots on the callback path.

Benny

> 
> --b.
> 
>> +	args->cbs_clp = clp;
>> +	clp->cl_cb_seq_nr++;
>> +	return 0;
>> +}
>> +
>> +static void
>> +nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
>> +{
>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
>> +
>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
>> +
>> +	/* FIXME: support multiple callback slots */
>> +	mutex_unlock(&clp->cl_cb_mutex);
>> +}
>> +#endif /* CONFIG_NFSD_V4_1 */
>> +
>>  /*
>>   * called with dp->dl_count inc'ed.
>>   */
>> diff --git a/include/linux/nfsd/state.h b/include/linux/nfsd/state.h
>> index f2783d4..0ea8c89 100644
>> --- a/include/linux/nfsd/state.h
>> +++ b/include/linux/nfsd/state.h
>> @@ -61,6 +61,10 @@ typedef struct {
>>  #define si_stateownerid   si_opaque.so_stateownerid
>>  #define si_fileid         si_opaque.so_fileid
>>  
>> +struct nfsd4_cb_sequence {
>> +	/* args/res */
>> +	struct nfs4_client	*cbs_clp;
>> +};
>>  
>>  struct nfs4_cb_recall {
>>  	u32			cbr_ident;
>> @@ -195,7 +199,8 @@ struct nfs4_client {
>>  	struct nfsd4_slot	cl_slot;	/* create_session slot */
>>  	u32			cl_exchange_flags;
>>  	struct nfs4_sessionid	cl_sessionid;
>> -
>> +	/* We currently support a single back channel with a single slot */
>> +	u32			cl_cb_seq_nr;
>>  	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
>>  	struct mutex		cl_cb_mutex;
>>  #endif /* CONFIG_NFSD_V4_1 */
>> -- 
>> 1.6.2.1
>>

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
> >> From: Andy Adamson <andros@netapp.com>
> >>
> >> Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1
> >>
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> [Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
> >> Signed-off-by: Andy Adamson <andros@netapp.com>
> >> [fixed indentation]
> >> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >> ---
> >>  fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
> >>  include/linux/nfsd/state.h |    7 ++-
> >>  2 files changed, 124 insertions(+), 1 deletions(-)
> >>
> >> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
> >> index 6b7ef87..7ada6b1 100644
> >> --- a/fs/nfsd/nfs4callback.c
> >> +++ b/fs/nfsd/nfs4callback.c
> >> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
> >>  	hdr->nops++;
> >>  }
> >>  
> >> +static void
> >> +encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
> >> +		   struct nfs4_cb_compound_hdr *hdr)
> >> +{
> >> +	__be32 *p;
> >> +
> >> +	if (hdr->minorversion == 0)
> >> +		return;
> >> +
> >> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
> >> +
> >> +	WRITE32(OP_CB_SEQUENCE);
> >> +#ifdef CONFIG_NFSD_V4_1
> >> +	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
> >> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
> >> +#endif /* CONFIG_NFSD_V4_1 */
> > 
> > This whole function should be under CONFIG_NFSD_V4_1.
> > 
> >> +	WRITE32(0);		/* slotid, always 0 */
> >> +	WRITE32(0);		/* highest slotid always 0 */
> >> +	WRITE32(0);		/* cachethis always 0 */
> >> +	WRITE32(0); /* FIXME: support referring_call_lists */
> >> +	hdr->nops++;
> >> +}
> >> +
> >>  static int
> >>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
> >>  {
> >> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
> >>  	return 0;
> >>  }
> >>  
> >> +/*
> >> + * Our current back channel implmentation supports a single backchannel
> >> + * with a single slot.
> >> + */
> >> +static int
> >> +decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
> >> +		   struct rpc_rqst *rqstp)
> >> +{
> >> +	struct nfs4_callback *cb =
> >> +		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
> >> +	struct nfs4_sessionid id;
> >> +	int status;
> >> +	u32 dummy;
> >> +	__be32 *p;
> >> +
> >> +	if (cb->cb_minorversion == 0)
> >> +		return 0;
> >> +
> >> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
> >> +	if (status)
> >> +		return status;
> >> +
> >> +	/*
> >> +	 * If the server returns different values for sessionID, slotID or
> >> +	 * sequence number, the server is looney tunes.
> >> +	 */
> >> +	status = -ESERVERFAULT;
> >> +
> >> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
> >> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
> >> +#ifdef CONFIG_NFSD_V4_1
> >> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
> >> +		   NFS4_MAX_SESSIONID_LEN)) {
> >> +		dprintk("%s Invalid session id\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	READ32(dummy);
> >> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
> >> +		dprintk("%s Invalid sequence number\n", __func__);
> >> +		goto out;
> >> +	}
> >> +#endif /* CONFIG_NFSD_V4_1 */
> > 
> > Ditto.
> > 
> >> +	READ32(dummy); 	/* slotid must be 0 */
> >> +	if (dummy != 0) {
> >> +		dprintk("%s Invalid slotid\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	READ32(dummy); 	/* highest slotid must be 0 */
> >> +	if (dummy != 0) {
> >> +		dprintk("%s Invalid highest slotid\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	READ32(dummy); 	/* target highest slotid must be 0 */
> >> +	if (dummy != 0) {
> >> +		dprintk("%s Invalid target highest slotid\n", __func__);
> >> +		goto out;
> >> +	}
> >> +	status = 0;
> >> +out:
> >> +	return status;
> >> +}
> >> +
> >> +
> >>  static int
> >>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
> >>  {
> >> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
> >>  	return;
> >>  }
> >>  
> >> +#if defined(CONFIG_NFSD_V4_1)
> >> +/*
> >> + * FIXME: cb_sequence should support referring call lists, cachethis, and
> >> + * multiple slots
> >> + */
> >> +static int
> >> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
> >> +{
> >> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> >> +
> >> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> >> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> >> +
> >> +	mutex_lock(&clp->cl_cb_mutex);
> > 
> > We shouldn't be holding a mutex across a callback.  Why is this needed?
> 
> Just a simple way to limit concurrency to 1 and not deal with
> multiple slots on the callback path.

OK.  We should work out some sort of asynchronous equivalent.  How is
the client side currently waiting on slots?

--b.

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Benny Halevy]

On Apr. 02, 2009, 21:51 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
>> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
>>>> From: Andy Adamson <andros@netapp.com>
>>>>
>>>> Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1
>>>>
>>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>>> [Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
>>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>>> [fixed indentation]
>>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
>>>> ---
>>>>  fs/nfsd/nfs4callback.c     |  118 ++++++++++++++++++++++++++++++++++++++++++++
>>>>  include/linux/nfsd/state.h |    7 ++-
>>>>  2 files changed, 124 insertions(+), 1 deletions(-)
>>>>
>>>> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
>>>> index 6b7ef87..7ada6b1 100644
>>>> --- a/fs/nfsd/nfs4callback.c
>>>> +++ b/fs/nfsd/nfs4callback.c
>>>> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_cb_recall *cb_rec,
>>>>  	hdr->nops++;
>>>>  }
>>>>  
>>>> +static void
>>>> +encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
>>>> +		   struct nfs4_cb_compound_hdr *hdr)
>>>> +{
>>>> +	__be32 *p;
>>>> +
>>>> +	if (hdr->minorversion == 0)
>>>> +		return;
>>>> +
>>>> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
>>>> +
>>>> +	WRITE32(OP_CB_SEQUENCE);
>>>> +#ifdef CONFIG_NFSD_V4_1
>>>> +	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
>>>> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
>>>> +#endif /* CONFIG_NFSD_V4_1 */
>>> This whole function should be under CONFIG_NFSD_V4_1.
>>>
>>>> +	WRITE32(0);		/* slotid, always 0 */
>>>> +	WRITE32(0);		/* highest slotid always 0 */
>>>> +	WRITE32(0);		/* cachethis always 0 */
>>>> +	WRITE32(0); /* FIXME: support referring_call_lists */
>>>> +	hdr->nops++;
>>>> +}
>>>> +
>>>>  static int
>>>>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
>>>>  {
>>>> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
>>>>  	return 0;
>>>>  }
>>>>  
>>>> +/*
>>>> + * Our current back channel implmentation supports a single backchannel
>>>> + * with a single slot.
>>>> + */
>>>> +static int
>>>> +decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
>>>> +		   struct rpc_rqst *rqstp)
>>>> +{
>>>> +	struct nfs4_callback *cb =
>>>> +		(struct nfs4_callback *)rqstp->rq_task->tk_client->cl_private;
>>>> +	struct nfs4_sessionid id;
>>>> +	int status;
>>>> +	u32 dummy;
>>>> +	__be32 *p;
>>>> +
>>>> +	if (cb->cb_minorversion == 0)
>>>> +		return 0;
>>>> +
>>>> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
>>>> +	if (status)
>>>> +		return status;
>>>> +
>>>> +	/*
>>>> +	 * If the server returns different values for sessionID, slotID or
>>>> +	 * sequence number, the server is looney tunes.
>>>> +	 */
>>>> +	status = -ESERVERFAULT;
>>>> +
>>>> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
>>>> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
>>>> +#ifdef CONFIG_NFSD_V4_1
>>>> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
>>>> +		   NFS4_MAX_SESSIONID_LEN)) {
>>>> +		dprintk("%s Invalid session id\n", __func__);
>>>> +		goto out;
>>>> +	}
>>>> +	READ32(dummy);
>>>> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
>>>> +		dprintk("%s Invalid sequence number\n", __func__);
>>>> +		goto out;
>>>> +	}
>>>> +#endif /* CONFIG_NFSD_V4_1 */
>>> Ditto.
>>>
>>>> +	READ32(dummy); 	/* slotid must be 0 */
>>>> +	if (dummy != 0) {
>>>> +		dprintk("%s Invalid slotid\n", __func__);
>>>> +		goto out;
>>>> +	}
>>>> +	READ32(dummy); 	/* highest slotid must be 0 */
>>>> +	if (dummy != 0) {
>>>> +		dprintk("%s Invalid highest slotid\n", __func__);
>>>> +		goto out;
>>>> +	}
>>>> +	READ32(dummy); 	/* target highest slotid must be 0 */
>>>> +	if (dummy != 0) {
>>>> +		dprintk("%s Invalid target highest slotid\n", __func__);
>>>> +		goto out;
>>>> +	}
>>>> +	status = 0;
>>>> +out:
>>>> +	return status;
>>>> +}
>>>> +
>>>> +
>>>>  static int
>>>>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
>>>>  {
>>>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
>>>>  	return;
>>>>  }
>>>>  
>>>> +#if defined(CONFIG_NFSD_V4_1)
>>>> +/*
>>>> + * FIXME: cb_sequence should support referring call lists, cachethis, and
>>>> + * multiple slots
>>>> + */
>>>> +static int
>>>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
>>>> +{
>>>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
>>>> +
>>>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
>>>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
>>>> +
>>>> +	mutex_lock(&clp->cl_cb_mutex);
>>> We shouldn't be holding a mutex across a callback.  Why is this needed?
>> Just a simple way to limit concurrency to 1 and not deal with
>> multiple slots on the callback path.
> 
> OK.  We should work out some sort of asynchronous equivalent.

OK.

> How is the client side currently waiting on slots?

Currently it has a single slot and a single
(NFS41_BC_MIN_CALLBACKS) preallocated rpc_rqst.
If it gets more calls than that, concurrently, it will
drop its end of the line using 
set_bit(XPRT_CLOSE_WAIT, &xprt->state)

(Ricardo, please keep me honest :)

Benny

> 
> --b.

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 10:20:08PM +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 21:51 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
> >> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
> >>>>  	return;
> >>>>  }
> >>>>  
> >>>> +#if defined(CONFIG_NFSD_V4_1)
> >>>> +/*
> >>>> + * FIXME: cb_sequence should support referring call lists, cachethis, and
> >>>> + * multiple slots
> >>>> + */
> >>>> +static int
> >>>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
> >>>> +{
> >>>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> >>>> +
> >>>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> >>>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> >>>> +
> >>>> +	mutex_lock(&clp->cl_cb_mutex);
> >>> We shouldn't be holding a mutex across a callback.  Why is this needed?
> >> Just a simple way to limit concurrency to 1 and not deal with
> >> multiple slots on the callback path.
> > 
> > OK.  We should work out some sort of asynchronous equivalent.
> 
> OK.
> 
> > How is the client side currently waiting on slots?
> 
> Currently it has a single slot and a single
> (NFS41_BC_MIN_CALLBACKS) preallocated rpc_rqst.
> If it gets more calls than that, concurrently, it will
> drop its end of the line using 
> set_bit(XPRT_CLOSE_WAIT, &xprt->state)
> 
> (Ricardo, please keep me honest :)

I was wondering about the rpc client as used on the nfs client, not the
nfs client/callback server.

--b.

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Benny Halevy]

On Apr. 02, 2009, 22:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Thu, Apr 02, 2009 at 10:20:08PM +0300, Benny Halevy wrote:
>> On Apr. 02, 2009, 21:51 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
>>>> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
>>>>>>  	return;
>>>>>>  }
>>>>>>  
>>>>>> +#if defined(CONFIG_NFSD_V4_1)
>>>>>> +/*
>>>>>> + * FIXME: cb_sequence should support referring call lists, cachethis, and
>>>>>> + * multiple slots
>>>>>> + */
>>>>>> +static int
>>>>>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
>>>>>> +{
>>>>>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
>>>>>> +
>>>>>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
>>>>>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
>>>>>> +
>>>>>> +	mutex_lock(&clp->cl_cb_mutex);
>>>>> We shouldn't be holding a mutex across a callback.  Why is this needed?
>>>> Just a simple way to limit concurrency to 1 and not deal with
>>>> multiple slots on the callback path.
>>> OK.  We should work out some sort of asynchronous equivalent.
>> OK.
>>
>>> How is the client side currently waiting on slots?
>> Currently it has a single slot and a single
>> (NFS41_BC_MIN_CALLBACKS) preallocated rpc_rqst.
>> If it gets more calls than that, concurrently, it will
>> drop its end of the line using 
>> set_bit(XPRT_CLOSE_WAIT, &xprt->state)
>>
>> (Ricardo, please keep me honest :)
> 
> I was wondering about the rpc client as used on the nfs client, not the

you mean rpc client as used on the nfs server? :)

> nfs client/callback server.
> 
> --b.

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 10:34:55PM +0300, Benny Halevy wrote:
> On Apr. 02, 2009, 22:27 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Thu, Apr 02, 2009 at 10:20:08PM +0300, Benny Halevy wrote:
> >> On Apr. 02, 2009, 21:51 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>> On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
> >>>> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client *clp)
> >>>>>>  	return;
> >>>>>>  }
> >>>>>>  
> >>>>>> +#if defined(CONFIG_NFSD_V4_1)
> >>>>>> +/*
> >>>>>> + * FIXME: cb_sequence should support referring call lists, cachethis, and
> >>>>>> + * multiple slots
> >>>>>> + */
> >>>>>> +static int
> >>>>>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct nfsd4_cb_sequence *args)
> >>>>>> +{
> >>>>>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> >>>>>> +
> >>>>>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> >>>>>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> >>>>>> +
> >>>>>> +	mutex_lock(&clp->cl_cb_mutex);
> >>>>> We shouldn't be holding a mutex across a callback.  Why is this needed?
> >>>> Just a simple way to limit concurrency to 1 and not deal with
> >>>> multiple slots on the callback path.
> >>> OK.  We should work out some sort of asynchronous equivalent.
> >> OK.
> >>
> >>> How is the client side currently waiting on slots?
> >> Currently it has a single slot and a single
> >> (NFS41_BC_MIN_CALLBACKS) preallocated rpc_rqst.
> >> If it gets more calls than that, concurrently, it will
> >> drop its end of the line using 
> >> set_bit(XPRT_CLOSE_WAIT, &xprt->state)
> >>
> >> (Ricardo, please keep me honest :)
> > 
> > I was wondering about the rpc client as used on the nfs client, not the
> 
> you mean rpc client as used on the nfs server? :)

I meant the rpc client on the nfs client: when the nfs client sends an
rpc on the forechannel it has to wait for a session slot.  How does it
do it (is it putting the rpc task on some queue?), and can the callback
client (on the nfs server) do the same thing?

--b.

RE: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Labiaga, Ricardo]

> -----Original Message-----
> From: J. Bruce Fields [mailto:bfields@fieldses.org]
> Sent: Thursday, April 02, 2009 1:54 PM
> To: Benny Halevy
> Cc: linux-nfs@vger.kernel.org; pnfs@linux-nfs.org; Labiaga, Ricardo
> Subject: Re: [PATCH v2 40/47] nfsd41: cb_sequence callback
> 
> On Thu, Apr 02, 2009 at 10:34:55PM +0300, Benny Halevy wrote:
> > On Apr. 02, 2009, 22:27 +0300, "J. Bruce Fields"
<bfields@fieldses.org>
> wrote:
> > > On Thu, Apr 02, 2009 at 10:20:08PM +0300, Benny Halevy wrote:
> > >> On Apr. 02, 2009, 21:51 +0300, "J. Bruce Fields"
> <bfields@fieldses.org> wrote:
> > >>> On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
> > >>>> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields"
> <bfields@fieldses.org> wrote:
> > >>>>>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client
> *clp)
> > >>>>>>  	return;
> > >>>>>>  }
> > >>>>>>
> > >>>>>> +#if defined(CONFIG_NFSD_V4_1)
> > >>>>>> +/*
> > >>>>>> + * FIXME: cb_sequence should support referring call lists,
> cachethis, and
> > >>>>>> + * multiple slots
> > >>>>>> + */
> > >>>>>> +static int
> > >>>>>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct
> nfsd4_cb_sequence *args)
> > >>>>>> +{
> > >>>>>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> > >>>>>> +
> > >>>>>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> > >>>>>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> > >>>>>> +
> > >>>>>> +	mutex_lock(&clp->cl_cb_mutex);
> > >>>>> We shouldn't be holding a mutex across a callback.  Why is
this
> needed?
> > >>>> Just a simple way to limit concurrency to 1 and not deal with
> > >>>> multiple slots on the callback path.
> > >>> OK.  We should work out some sort of asynchronous equivalent.
> > >> OK.
> > >>
> > >>> How is the client side currently waiting on slots?
> > >> Currently it has a single slot and a single
> > >> (NFS41_BC_MIN_CALLBACKS) preallocated rpc_rqst.
> > >> If it gets more calls than that, concurrently, it will
> > >> drop its end of the line using
> > >> set_bit(XPRT_CLOSE_WAIT, &xprt->state)
> > >>
> > >> (Ricardo, please keep me honest :)
> > >
> > > I was wondering about the rpc client as used on the nfs client,
not
> the
> >
> > you mean rpc client as used on the nfs server? :)
> 
> I meant the rpc client on the nfs client: when the nfs client sends an
> rpc on the forechannel it has to wait for a session slot.  How does it
> do it (is it putting the rpc task on some queue?), and can the
callback
> client (on the nfs server) do the same thing?
> 

Yes, the NFS client uses a slot table for the forechannel 'struct
nfs4_slot_table' and a slot table for the backchannel.  Tasks sleep on
an rpc_wait_queue if there are no available slots.  Used/ unused slots
are tracked with a bit map array.  When the reply is received on a slot,
the next available task is awaken.

Yes, the callback client can certainly do the same thing.  Today, the
Linux client backchannel only advertises a single slot (need to check
what Solaris does).  So against Linux, having more than one slot doesn't
buy the server much right now.

Is this something that can be addressed as an enhancement later on, or
do you need this implemented right away?

- ricardo

Re: [PATCH v2 40/47] nfsd41: cb_sequence callback

[J. Bruce Fields]

On Thu, Apr 02, 2009 at 06:06:55PM -0700, Labiaga, Ricardo wrote:
> Yes, the NFS client uses a slot table for the forechannel 'struct
> nfs4_slot_table' and a slot table for the backchannel.  Tasks sleep on
> an rpc_wait_queue if there are no available slots.  Used/ unused slots
> are tracked with a bit map array.  When the reply is received on a slot,
> the next available task is awaken.
> 
> Yes, the callback client can certainly do the same thing.  Today, the
> Linux client backchannel only advertises a single slot (need to check
> what Solaris does).  So against Linux, having more than one slot doesn't
> buy the server much right now.
> 
> Is this something that can be addressed as an enhancement later on, or
> do you need this implemented right away?

I'm dropping the backchannel patches for 2.6.30, so it isn't urgent, but
this is something to work on over the next month or two.

I'd just like an asynchronous solution, so no big mutexes.  I'm fine
with the single-slot limitation if it helps, but maybe the simplest
thing will be just to imitate the nfs-client code?

--b.

RE: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Labiaga, Ricardo]

> -----Original Message-----
> From: J. Bruce Fields [mailto:bfields@fieldses.org]
> Sent: Thursday, April 02, 2009 7:34 PM
> To: Labiaga, Ricardo
> Cc: Benny Halevy; linux-nfs@vger.kernel.org; pnfs@linux-nfs.org
> Subject: Re: [PATCH v2 40/47] nfsd41: cb_sequence callback
> 
> On Thu, Apr 02, 2009 at 06:06:55PM -0700, Labiaga, Ricardo wrote:
> > Yes, the NFS client uses a slot table for the forechannel 'struct
> > nfs4_slot_table' and a slot table for the backchannel.  Tasks sleep
on
> > an rpc_wait_queue if there are no available slots.  Used/ unused
slots
> > are tracked with a bit map array.  When the reply is received on a
slot,
> > the next available task is awaken.
> >
> > Yes, the callback client can certainly do the same thing.  Today,
the
> > Linux client backchannel only advertises a single slot (need to
check
> > what Solaris does).  So against Linux, having more than one slot
doesn't
> > buy the server much right now.
> >
> > Is this something that can be addressed as an enhancement later on,
or
> > do you need this implemented right away?
> 
> I'm dropping the backchannel patches for 2.6.30, so it isn't urgent,
but
> this is something to work on over the next month or two.
> 
> I'd just like an asynchronous solution, so no big mutexes.  I'm fine
> with the single-slot limitation if it helps, but maybe the simplest
> thing will be just to imitate the nfs-client code?
> 

Sounds good, I can work on this.

- ricardo

RE: [PATCH v2 40/47] nfsd41: cb_sequence callback

[Labiaga, Ricardo]

> -----Original Message-----
> From: Benny Halevy [mailto:bhalevy@panasas.com]
> Sent: Thursday, April 02, 2009 12:20 PM
> To: J. Bruce Fields
> Cc: linux-nfs@vger.kernel.org; pnfs@linux-nfs.org; Labiaga, Ricardo
> Subject: Re: [PATCH v2 40/47] nfsd41: cb_sequence callback
> 
> On Apr. 02, 2009, 21:51 +0300, "J. Bruce Fields"
<bfields@fieldses.org>
> wrote:
> > On Thu, Apr 02, 2009 at 11:47:24AM +0300, Benny Halevy wrote:
> >> On Apr. 01, 2009, 7:39 +0300, "J. Bruce Fields"
<bfields@fieldses.org>
> wrote:
> >>> On Sat, Mar 28, 2009 at 11:34:41AM +0300, Benny Halevy wrote:
> >>>> From: Andy Adamson <andros@netapp.com>
> >>>>
> >>>> Implement the cb_sequence callback conforming to
draft-ietf-nfsv4-
> minorversion1
> >>>>
> >>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >>>> [Rework the back channel xdr using the shared v4.0 and v4.1
> framework.]
> >>>> Signed-off-by: Andy Adamson <andros@netapp.com>
> >>>> [fixed indentation]
> >>>> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
> >>>> ---
> >>>>  fs/nfsd/nfs4callback.c     |  118
> ++++++++++++++++++++++++++++++++++++++++++++
> >>>>  include/linux/nfsd/state.h |    7 ++-
> >>>>  2 files changed, 124 insertions(+), 1 deletions(-)
> >>>>
> >>>> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
> >>>> index 6b7ef87..7ada6b1 100644
> >>>> --- a/fs/nfsd/nfs4callback.c
> >>>> +++ b/fs/nfsd/nfs4callback.c
> >>>> @@ -255,6 +255,29 @@ encode_cb_recall(struct xdr_stream *xdr,
struct
> nfs4_cb_recall *cb_rec,
> >>>>  	hdr->nops++;
> >>>>  }
> >>>>
> >>>> +static void
> >>>> +encode_cb_sequence(struct xdr_stream *xdr, struct
nfsd4_cb_sequence
> *args,
> >>>> +		   struct nfs4_cb_compound_hdr *hdr)
> >>>> +{
> >>>> +	__be32 *p;
> >>>> +
> >>>> +	if (hdr->minorversion == 0)
> >>>> +		return;
> >>>> +
> >>>> +	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
> >>>> +
> >>>> +	WRITE32(OP_CB_SEQUENCE);
> >>>> +#ifdef CONFIG_NFSD_V4_1
> >>>> +	WRITEMEM(args->cbs_clp->cl_sessionid.data,
> NFS4_MAX_SESSIONID_LEN);
> >>>> +	WRITE32(args->cbs_clp->cl_cb_seq_nr);
> >>>> +#endif /* CONFIG_NFSD_V4_1 */
> >>> This whole function should be under CONFIG_NFSD_V4_1.
> >>>
> >>>> +	WRITE32(0);		/* slotid, always 0 */
> >>>> +	WRITE32(0);		/* highest slotid always 0 */
> >>>> +	WRITE32(0);		/* cachethis always 0 */
> >>>> +	WRITE32(0); /* FIXME: support referring_call_lists */
> >>>> +	hdr->nops++;
> >>>> +}
> >>>> +
> >>>>  static int
> >>>>  nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
> >>>>  {
> >>>> @@ -319,6 +342,69 @@ decode_cb_op_hdr(struct xdr_stream *xdr,
enum
> nfs_opnum4 expected)
> >>>>  	return 0;
> >>>>  }
> >>>>
> >>>> +/*
> >>>> + * Our current back channel implmentation supports a single
> backchannel
> >>>> + * with a single slot.
> >>>> + */
> >>>> +static int
> >>>> +decode_cb_sequence(struct xdr_stream *xdr, struct
nfsd4_cb_sequence
> *res,
> >>>> +		   struct rpc_rqst *rqstp)
> >>>> +{
> >>>> +	struct nfs4_callback *cb =
> >>>> +		(struct nfs4_callback
*)rqstp->rq_task->tk_client-
> >cl_private;
> >>>> +	struct nfs4_sessionid id;
> >>>> +	int status;
> >>>> +	u32 dummy;
> >>>> +	__be32 *p;
> >>>> +
> >>>> +	if (cb->cb_minorversion == 0)
> >>>> +		return 0;
> >>>> +
> >>>> +	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
> >>>> +	if (status)
> >>>> +		return status;
> >>>> +
> >>>> +	/*
> >>>> +	 * If the server returns different values for sessionID,
slotID
> or
> >>>> +	 * sequence number, the server is looney tunes.
> >>>> +	 */
> >>>> +	status = -ESERVERFAULT;
> >>>> +
> >>>> +	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
> >>>> +	COPYMEM(id.data, NFS4_MAX_SESSIONID_LEN);
> >>>> +#ifdef CONFIG_NFSD_V4_1
> >>>> +	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
> >>>> +		   NFS4_MAX_SESSIONID_LEN)) {
> >>>> +		dprintk("%s Invalid session id\n", __func__);
> >>>> +		goto out;
> >>>> +	}
> >>>> +	READ32(dummy);
> >>>> +	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
> >>>> +		dprintk("%s Invalid sequence number\n",
__func__);
> >>>> +		goto out;
> >>>> +	}
> >>>> +#endif /* CONFIG_NFSD_V4_1 */
> >>> Ditto.
> >>>
> >>>> +	READ32(dummy); 	/* slotid must be 0 */
> >>>> +	if (dummy != 0) {
> >>>> +		dprintk("%s Invalid slotid\n", __func__);
> >>>> +		goto out;
> >>>> +	}
> >>>> +	READ32(dummy); 	/* highest slotid must be 0 */
> >>>> +	if (dummy != 0) {
> >>>> +		dprintk("%s Invalid highest slotid\n",
__func__);
> >>>> +		goto out;
> >>>> +	}
> >>>> +	READ32(dummy); 	/* target highest slotid must be 0 */
> >>>> +	if (dummy != 0) {
> >>>> +		dprintk("%s Invalid target highest slotid\n",
__func__);
> >>>> +		goto out;
> >>>> +	}
> >>>> +	status = 0;
> >>>> +out:
> >>>> +	return status;
> >>>> +}
> >>>> +
> >>>> +
> >>>>  static int
> >>>>  nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
> >>>>  {
> >>>> @@ -503,6 +589,38 @@ nfsd4_probe_callback(struct nfs4_client
*clp)
> >>>>  	return;
> >>>>  }
> >>>>
> >>>> +#if defined(CONFIG_NFSD_V4_1)
> >>>> +/*
> >>>> + * FIXME: cb_sequence should support referring call lists,
> cachethis, and
> >>>> + * multiple slots
> >>>> + */
> >>>> +static int
> >>>> +nfs41_cb_sequence_setup(struct nfs4_client *clp, struct
> nfsd4_cb_sequence *args)
> >>>> +{
> >>>> +	u32 *ptr = (u32 *)clp->cl_sessionid.data;
> >>>> +
> >>>> +	dprintk("%s: %u:%u:%u:%u\n", __func__,
> >>>> +		ptr[0], ptr[1], ptr[2], ptr[3]);
> >>>> +
> >>>> +	mutex_lock(&clp->cl_cb_mutex);
> >>> We shouldn't be holding a mutex across a callback.  Why is this
> needed?
> >> Just a simple way to limit concurrency to 1 and not deal with
> >> multiple slots on the callback path.
> >
> > OK.  We should work out some sort of asynchronous equivalent.
> 
> OK.
> 
> > How is the client side currently waiting on slots?
> 
> Currently it has a single slot and a single
> (NFS41_BC_MIN_CALLBACKS) preallocated rpc_rqst.
> If it gets more calls than that, concurrently, it will
> drop its end of the line using
> set_bit(XPRT_CLOSE_WAIT, &xprt->state)

That's correct.  The assumption here is that the server had violated the
session contract and it should reestablish it.

- ricardo

> (Ricardo, please keep me honest :)
> 
> Benny
> 
> >
> > --b.

[PATCH v2 41/47] nfsd41: introduce nfs4_cb_call_sync for nfs4 and nfs41

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

For nfs41 callbacks we need to setup the cb_sequence args and
process the result.  This will be called by code common to
nfs4 and nfs41.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   50 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 50 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 7ada6b1..dd3b739 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -92,6 +92,15 @@ enum nfs_cb_opnum4 {
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
+struct nfs4_rpc_args {
+	void                     *args_op;
+	struct nfsd4_cb_sequence *args_seq;
+};
+
+struct nfs4_rpc_res {
+	struct nfsd4_cb_sequence *res_seq;
+};
+
 /*
 * Generic encode routines from fs/nfs/nfs4xdr.c
 */
@@ -589,6 +598,12 @@ nfsd4_probe_callback(struct nfs4_client *clp)
 	return;
 }
 
+static int _nfsd4_cb_sync(struct nfs4_client *clp,
+			  const struct rpc_message *msg, int flags)
+{
+	return rpc_call_sync(clp->cl_callback.cb_client, msg, RPC_TASK_SOFT);
+}
+
 #if defined(CONFIG_NFSD_V4_1)
 /*
  * FIXME: cb_sequence should support referring call lists, cachethis, and
@@ -619,6 +634,41 @@ nfs41_cb_sequence_done(struct nfs4_client *clp, struct nfsd4_cb_sequence *res)
 	/* FIXME: support multiple callback slots */
 	mutex_unlock(&clp->cl_cb_mutex);
 }
+
+static int _nfsd41_cb_sync(struct nfs4_client *clp,
+			   struct rpc_message *msg, int flags)
+{
+	struct nfsd4_cb_sequence seq;
+	struct nfs4_rpc_args *args;
+	struct nfs4_rpc_res res;
+	int status;
+
+	args = msg->rpc_argp;
+	args->args_seq = &seq;
+
+	res.res_seq = &seq;
+	msg->rpc_resp = &res;
+
+	nfs41_cb_sequence_setup(clp, &seq);
+	status = _nfsd4_cb_sync(clp, msg, flags);
+	nfs41_cb_sequence_done(clp, &seq);
+
+	return status;
+}
+
+static int nfsd4_cb_sync(struct nfs4_client *clp,
+			 struct rpc_message *msg, int flags)
+{
+	return clp->cl_callback.cb_minorversion ?
+		_nfsd41_cb_sync(clp, msg, flags) :
+		_nfsd4_cb_sync(clp, msg, flags);
+}
+#else  /* CONFIG_NFSD_V4_1 */
+static int nfsd4_cb_sync(struct nfs4_client *clp,
+			 struct rpc_message *msg, int flags)
+{
+	return _nfsd4_cb_sync(clp, msg, flags);
+}
 #endif /* CONFIG_NFSD_V4_1 */
 
 /*
-- 
1.6.2.1

[PATCH v2 42/47] nfsd41: cb_recall callback

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Implement the cb_recall callback conforming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

Signed-off-by: Ricardo Labiaga <ricardo.labiaga@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[Share v4.0 and v4.1 back channel xdr]
Signed-off-by: Andy Adamson <andros@netapp.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4callback.c |   25 +++++++++++++++++++------
 1 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index dd3b739..0be75e0 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -298,9 +298,11 @@ nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *args)
+nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p,
+		       struct nfs4_rpc_args *rpc_args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_cb_recall *args = rpc_args->args_op;
 	struct nfs4_callback *cb =
 		(struct nfs4_callback *)req->rq_task->tk_client->cl_private;
 	struct nfs4_cb_compound_hdr hdr = {
@@ -310,6 +312,7 @@ nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_cb_recall *a
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
+	encode_cb_sequence(&xdr, rpc_args->args_seq, &hdr);
 	encode_cb_recall(&xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 	return 0;
@@ -421,7 +424,8 @@ nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
+nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
+		       struct nfs4_rpc_res *rpc_res)
 {
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr;
@@ -431,6 +435,9 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
+	status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
+	if (status)
+		goto out;
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;
@@ -678,19 +685,23 @@ void
 nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
-	struct rpc_clnt *clnt = clp->cl_callback.cb_client;
 	struct nfs4_cb_recall *cbr = &dp->dl_recall;
+	struct nfs4_rpc_args args = {
+		.args_op = cbr,
+	};
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = cbr,
+		.rpc_argp = &args,
 	};
 	int retries = 1;
 	int status = 0;
 
+	dprintk("%s: dp %p\n", __func__, dp);
+
 	cbr->cbr_trunc = 0; /* XXX need to implement truncate optimization */
 	cbr->cbr_dp = dp;
 
-	status = rpc_call_sync(clnt, &msg, RPC_TASK_SOFT);
+	status = nfsd4_cb_sync(clp, &msg, RPC_TASK_SOFT);
 	while (retries--) {
 		switch (status) {
 			case -EIO:
@@ -705,13 +716,15 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 				goto out_put_cred;
 		}
 		ssleep(2);
-		status = rpc_call_sync(clnt, &msg, RPC_TASK_SOFT);
+		status = nfsd4_cb_sync(clp, &msg, RPC_TASK_SOFT);
 	}
 out_put_cred:
 	/*
 	 * Success or failure, now we're either waiting for lease expiration
 	 * or deleg_return.
 	 */
+	dprintk("%s: dp %p dl_flock %p dl_count %d\n",
+		__func__, dp, dp->dl_flock, atomic_read(&dp->dl_count));
 	put_nfs4_client(clp);
 	nfs4_put_delegation(dp);
 	return;
-- 
1.6.2.1

[PATCH v2 43/47] nfsd41: pass writable attrs mask to nfsd4_decode_fattr

[Benny Halevy]

In preparation for EXCLUSIVE4_1

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c |   21 +++++++++++++++------
 1 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a1af3a3..96f3eeb 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -251,9 +251,14 @@ nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 	DECODE_TAIL;
 }
 
+static u32 nfsd_attrmask[] = {
+	NFSD_WRITEABLE_ATTRS_WORD0,
+	NFSD_WRITEABLE_ATTRS_WORD1
+};
+
 static __be32
-nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, struct iattr *iattr,
-    struct nfs4_acl **acl)
+nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
+		   struct iattr *iattr, struct nfs4_acl **acl)
 {
 	int expected_len, len = 0;
 	u32 dummy32;
@@ -271,7 +276,7 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, struct iattr *ia
 	 */
 	if ((bmval[0] & ~NFSD_SUPPORTED_ATTRS_WORD0) || (bmval[1] & ~NFSD_SUPPORTED_ATTRS_WORD1))
 		return nfserr_attrnotsupp;
-	if ((bmval[0] & ~NFSD_WRITEABLE_ATTRS_WORD0) || (bmval[1] & ~NFSD_WRITEABLE_ATTRS_WORD1))
+	if ((bmval[0] & ~writable[0]) || (bmval[1] & ~writable[1]))
 		return nfserr_inval;
 
 	READ_BUF(4);
@@ -499,7 +504,9 @@ nfsd4_decode_create(struct nfsd4_compoundargs *argp, struct nfsd4_create *create
 	if ((status = check_filename(create->cr_name, create->cr_namelen, nfserr_inval)))
 		return status;
 
-	if ((status = nfsd4_decode_fattr(argp, create->cr_bmval, &create->cr_iattr, &create->cr_acl)))
+	status = nfsd4_decode_fattr(argp, create->cr_bmval, nfsd_attrmask,
+				    &create->cr_iattr, &create->cr_acl);
+	if (status)
 		goto out;
 
 	DECODE_TAIL;
@@ -660,7 +667,9 @@ nfsd4_decode_open(struct nfsd4_compoundargs *argp, struct nfsd4_open *open)
 		switch (open->op_createmode) {
 		case NFS4_CREATE_UNCHECKED:
 		case NFS4_CREATE_GUARDED:
-			if ((status = nfsd4_decode_fattr(argp, open->op_bmval, &open->op_iattr, &open->op_acl)))
+			status = nfsd4_decode_fattr(argp, open->op_bmval,
+				nfsd_attrmask, &open->op_iattr, &open->op_acl);
+			if (status)
 				goto out;
 			break;
 		case NFS4_CREATE_EXCLUSIVE:
@@ -859,7 +868,7 @@ nfsd4_decode_setattr(struct nfsd4_compoundargs *argp, struct nfsd4_setattr *seta
 	status = nfsd4_decode_stateid(argp, &setattr->sa_stateid);
 	if (status)
 		return status;
-	return nfsd4_decode_fattr(argp, setattr->sa_bmval,
+	return nfsd4_decode_fattr(argp, setattr->sa_bmval, nfsd_attrmask,
 				  &setattr->sa_iattr, &setattr->sa_acl);
 }
 
-- 
1.6.2.1

[PATCH v2 44/47] nfsd41: support for 3-word long attribute bitmask

[Benny Halevy]

From: Andy Adamson <andros@netapp.com>

Also, use client minorversion to generate supported attrs

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   15 +++++++----
 fs/nfsd/nfs4xdr.c         |   59 +++++++++++++++++++++++++++++++++++---------
 include/linux/nfsd/nfsd.h |   48 +++++++++++++++++++++++++++++++++++-
 include/linux/nfsd/xdr4.h |   12 ++++----
 4 files changed, 108 insertions(+), 26 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 8b4bf5e..f3b560f 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -460,8 +460,9 @@ nfsd4_getattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (getattr->ga_bmval[1] & NFSD_WRITEONLY_ATTRS_WORD1)
 		return nfserr_inval;
 
-	getattr->ga_bmval[0] &= NFSD_SUPPORTED_ATTRS_WORD0;
-	getattr->ga_bmval[1] &= NFSD_SUPPORTED_ATTRS_WORD1;
+	getattr->ga_bmval[0] &= nfsd_suppattrs0(cstate->minorversion);
+	getattr->ga_bmval[1] &= nfsd_suppattrs1(cstate->minorversion);
+	getattr->ga_bmval[2] &= nfsd_suppattrs2(cstate->minorversion);
 
 	getattr->ga_fhp = &cstate->current_fh;
 	return nfs_ok;
@@ -556,8 +557,9 @@ nfsd4_readdir(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (readdir->rd_bmval[1] & NFSD_WRITEONLY_ATTRS_WORD1)
 		return nfserr_inval;
 
-	readdir->rd_bmval[0] &= NFSD_SUPPORTED_ATTRS_WORD0;
-	readdir->rd_bmval[1] &= NFSD_SUPPORTED_ATTRS_WORD1;
+	readdir->rd_bmval[0] &= nfsd_suppattrs0(cstate->minorversion);
+	readdir->rd_bmval[1] &= nfsd_suppattrs1(cstate->minorversion);
+	readdir->rd_bmval[2] &= nfsd_suppattrs2(cstate->minorversion);
 
 	if ((cookie > ~(u32)0) || (cookie == 1) || (cookie == 2) ||
 	    (cookie == 0 && memcmp(readdir->rd_verf.data, zeroverf.data, NFS4_VERIFIER_SIZE)))
@@ -754,8 +756,9 @@ _nfsd4_verify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (status)
 		return status;
 
-	if ((verify->ve_bmval[0] & ~NFSD_SUPPORTED_ATTRS_WORD0)
-	    || (verify->ve_bmval[1] & ~NFSD_SUPPORTED_ATTRS_WORD1))
+	if ((verify->ve_bmval[0] & ~nfsd_suppattrs0(cstate->minorversion))
+	    || (verify->ve_bmval[1] & ~nfsd_suppattrs1(cstate->minorversion))
+	    || (verify->ve_bmval[2] & ~nfsd_suppattrs2(cstate->minorversion)))
 		return nfserr_attrnotsupp;
 	if ((verify->ve_bmval[0] & FATTR4_WORD0_RDATTR_ERROR)
 	    || (verify->ve_bmval[1] & NFSD_WRITEONLY_ATTRS_WORD1))
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 96f3eeb..079626c 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -236,6 +236,7 @@ nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 
 	bmval[0] = 0;
 	bmval[1] = 0;
+	bmval[2] = 0;
 
 	READ_BUF(4);
 	READ32(bmlen);
@@ -247,13 +248,16 @@ nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 		READ32(bmval[0]);
 	if (bmlen > 1)
 		READ32(bmval[1]);
+	if (bmlen > 2)
+		READ32(bmval[2]);
 
 	DECODE_TAIL;
 }
 
 static u32 nfsd_attrmask[] = {
 	NFSD_WRITEABLE_ATTRS_WORD0,
-	NFSD_WRITEABLE_ATTRS_WORD1
+	NFSD_WRITEABLE_ATTRS_WORD1,
+	NFSD_WRITEABLE_ATTRS_WORD2
 };
 
 static __be32
@@ -274,9 +278,12 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
 	 * According to spec, unsupported attributes return ERR_ATTRNOTSUPP;
 	 * read-only attributes return ERR_INVAL.
 	 */
-	if ((bmval[0] & ~NFSD_SUPPORTED_ATTRS_WORD0) || (bmval[1] & ~NFSD_SUPPORTED_ATTRS_WORD1))
+	if ((bmval[0] & ~nfsd_suppattrs0(argp->minorversion)) ||
+	    (bmval[1] & ~nfsd_suppattrs1(argp->minorversion)) ||
+	    (bmval[2] & ~nfsd_suppattrs2(argp->minorversion)))
 		return nfserr_attrnotsupp;
-	if ((bmval[0] & ~writable[0]) || (bmval[1] & ~writable[1]))
+	if ((bmval[0] & ~writable[0]) || (bmval[1] & ~writable[1]) ||
+	    (bmval[2] & ~writable[2]))
 		return nfserr_inval;
 
 	READ_BUF(4);
@@ -411,6 +418,7 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
 			goto xdr_error;
 		}
 	}
+	BUG_ON(bmval[2]);	/* no such writeable attr supported yet */
 	if (len != expected_len)
 		goto xdr_error;
 
@@ -1732,6 +1740,7 @@ nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
 {
 	u32 bmval0 = bmval[0];
 	u32 bmval1 = bmval[1];
+	u32 bmval2 = bmval[2];
 	struct kstat stat;
 	struct svc_fh tempfh;
 	struct kstatfs statfs;
@@ -1745,12 +1754,16 @@ nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
 	int err;
 	int aclsupport = 0;
 	struct nfs4_acl *acl = NULL;
+	struct nfsd4_compoundres *resp = rqstp->rq_resp;
+	u32 minorversion = resp->cstate.minorversion;
 
 	BUG_ON(bmval1 & NFSD_WRITEONLY_ATTRS_WORD1);
-	BUG_ON(bmval0 & ~NFSD_SUPPORTED_ATTRS_WORD0);
-	BUG_ON(bmval1 & ~NFSD_SUPPORTED_ATTRS_WORD1);
+	BUG_ON(bmval0 & ~nfsd_suppattrs0(minorversion));
+	BUG_ON(bmval1 & ~nfsd_suppattrs1(minorversion));
+	BUG_ON(bmval2 & ~nfsd_suppattrs2(minorversion));
 
 	if (exp->ex_fslocs.migrated) {
+		BUG_ON(bmval[2]);
 		status = fattr_handle_absent_fs(&bmval0, &bmval1, &rdattr_err);
 		if (status)
 			goto out;
@@ -1796,22 +1809,42 @@ nfsd4_encode_fattr(struct svc_fh *fhp, struct svc_export *exp,
 	if ((buflen -= 16) < 0)
 		goto out_resource;
 
-	WRITE32(2);
-	WRITE32(bmval0);
-	WRITE32(bmval1);
+	if (unlikely(bmval2)) {
+		WRITE32(3);
+		WRITE32(bmval0);
+		WRITE32(bmval1);
+		WRITE32(bmval2);
+	} else if (likely(bmval1)) {
+		WRITE32(2);
+		WRITE32(bmval0);
+		WRITE32(bmval1);
+	} else {
+		WRITE32(1);
+		WRITE32(bmval0);
+	}
 	attrlenp = p++;                /* to be backfilled later */
 
 	if (bmval0 & FATTR4_WORD0_SUPPORTED_ATTRS) {
-		u32 word0 = NFSD_SUPPORTED_ATTRS_WORD0;
+		u32 word0 = nfsd_suppattrs0(minorversion);
+		u32 word1 = nfsd_suppattrs1(minorversion);
+		u32 word2 = nfsd_suppattrs2(minorversion);
+
 		if ((buflen -= 12) < 0)
 			goto out_resource;
 		if (!aclsupport)
 			word0 &= ~FATTR4_WORD0_ACL;
 		if (!exp->ex_fslocs.locations)
 			word0 &= ~FATTR4_WORD0_FS_LOCATIONS;
-		WRITE32(2);
-		WRITE32(word0);
-		WRITE32(NFSD_SUPPORTED_ATTRS_WORD1);
+		if (!word2) {
+			WRITE32(2);
+			WRITE32(word0);
+			WRITE32(word1);
+		} else {
+			WRITE32(3);
+			WRITE32(word0);
+			WRITE32(word1);
+			WRITE32(word2);
+		}
 	}
 	if (bmval0 & FATTR4_WORD0_TYPE) {
 		if ((buflen -= 4) < 0)
@@ -2121,6 +2154,8 @@ out_acl:
 		}
 		WRITE64(stat.ino);
 	}
+	BUG_ON(bmval2);	/* FIXME: not implemented yet */
+
 	*attrlenp = htonl((char *)p - (char *)attrlenp - 4);
 	*countp = p - buffer;
 	status = nfs_ok;
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index d0fe9b4..3dccb76 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -345,7 +345,7 @@ extern struct timeval	nfssvc_boot;
  *    TIME_BACKUP   (unlikely to be supported any time soon)
  *    TIME_CREATE   (unlikely to be supported any time soon)
  */
-#define NFSD_SUPPORTED_ATTRS_WORD0                                                          \
+#define NFSD4_SUPPORTED_ATTRS_WORD0                                                         \
 (FATTR4_WORD0_SUPPORTED_ATTRS   | FATTR4_WORD0_TYPE         | FATTR4_WORD0_FH_EXPIRE_TYPE   \
  | FATTR4_WORD0_CHANGE          | FATTR4_WORD0_SIZE         | FATTR4_WORD0_LINK_SUPPORT     \
  | FATTR4_WORD0_SYMLINK_SUPPORT | FATTR4_WORD0_NAMED_ATTR   | FATTR4_WORD0_FSID             \
@@ -357,7 +357,7 @@ extern struct timeval	nfssvc_boot;
  | FATTR4_WORD0_MAXFILESIZE     | FATTR4_WORD0_MAXLINK      | FATTR4_WORD0_MAXNAME          \
  | FATTR4_WORD0_MAXREAD         | FATTR4_WORD0_MAXWRITE     | FATTR4_WORD0_ACL)
 
-#define NFSD_SUPPORTED_ATTRS_WORD1                                                          \
+#define NFSD4_SUPPORTED_ATTRS_WORD1                                                         \
 (FATTR4_WORD1_MODE              | FATTR4_WORD1_NO_TRUNC     | FATTR4_WORD1_NUMLINKS         \
  | FATTR4_WORD1_OWNER	        | FATTR4_WORD1_OWNER_GROUP  | FATTR4_WORD1_RAWDEV           \
  | FATTR4_WORD1_SPACE_AVAIL     | FATTR4_WORD1_SPACE_FREE   | FATTR4_WORD1_SPACE_TOTAL      \
@@ -365,11 +365,54 @@ extern struct timeval	nfssvc_boot;
  | FATTR4_WORD1_TIME_DELTA   | FATTR4_WORD1_TIME_METADATA    \
  | FATTR4_WORD1_TIME_MODIFY     | FATTR4_WORD1_TIME_MODIFY_SET | FATTR4_WORD1_MOUNTED_ON_FILEID)
 
+#define NFSD4_SUPPORTED_ATTRS_WORD2 0
+
+#define NFSD4_1_SUPPORTED_ATTRS_WORD0 \
+	NFSD4_SUPPORTED_ATTRS_WORD0
+
+#define NFSD4_1_SUPPORTED_ATTRS_WORD1 \
+	NFSD4_SUPPORTED_ATTRS_WORD1
+
+#define NFSD4_1_SUPPORTED_ATTRS_WORD2 \
+	NFSD4_SUPPORTED_ATTRS_WORD2
+
 #if defined(CONFIG_NFSD_V4_1)
 
 /* The percent of nr_free_buffer_pages used by the V4.1 server DRC */
 #define NFSD_DRC_SIZE_SHIFT	7
 
+static inline u32 nfsd_suppattrs0(u32 minorversion)
+{
+	return minorversion ? NFSD4_1_SUPPORTED_ATTRS_WORD0
+			    : NFSD4_SUPPORTED_ATTRS_WORD0;
+}
+
+static inline u32 nfsd_suppattrs1(u32 minorversion)
+{
+	return minorversion ? NFSD4_1_SUPPORTED_ATTRS_WORD1
+			    : NFSD4_SUPPORTED_ATTRS_WORD1;
+}
+
+static inline u32 nfsd_suppattrs2(u32 minorversion)
+{
+	return minorversion ? NFSD4_1_SUPPORTED_ATTRS_WORD2
+			    : NFSD4_SUPPORTED_ATTRS_WORD2;
+}
+#else  /* CONFIG_NFSD_V4_1 */
+static inline u32 nfsd_suppattrs0(u32 minorversion)
+{
+	return NFSD4_SUPPORTED_ATTRS_WORD0;
+}
+
+static inline u32 nfsd_suppattrs1(u32 minorversion)
+{
+	return NFSD4_SUPPORTED_ATTRS_WORD1;
+}
+
+static inline u32 nfsd_suppattrs2(u32 minorversion)
+{
+	return NFSD4_SUPPORTED_ATTRS_WORD2;
+}
 #endif /* CONFIG_NFSD_V4_1 */
 
 /* These will return ERR_INVAL if specified in GETATTR or READDIR. */
@@ -382,6 +425,7 @@ extern struct timeval	nfssvc_boot;
 #define NFSD_WRITEABLE_ATTRS_WORD1                                                          \
 (FATTR4_WORD1_MODE              | FATTR4_WORD1_OWNER         | FATTR4_WORD1_OWNER_GROUP     \
  | FATTR4_WORD1_TIME_ACCESS_SET | FATTR4_WORD1_TIME_MODIFY_SET)
+#define NFSD_WRITEABLE_ATTRS_WORD2 0
 
 #endif /* CONFIG_NFSD_V4 */
 
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index 881ff67..d7a8050 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -101,7 +101,7 @@ struct nfsd4_create {
 			u32 specdata2;
 		} dev;    /* NF4BLK, NF4CHR */
 	} u;
-	u32		cr_bmval[2];        /* request */
+	u32		cr_bmval[3];        /* request */
 	struct iattr	cr_iattr;           /* request */
 	struct nfsd4_change_info  cr_cinfo; /* response */
 	struct nfs4_acl *cr_acl;
@@ -116,7 +116,7 @@ struct nfsd4_delegreturn {
 };
 
 struct nfsd4_getattr {
-	u32		ga_bmval[2];        /* request */
+	u32		ga_bmval[3];        /* request */
 	struct svc_fh	*ga_fhp;            /* response */
 };
 
@@ -217,7 +217,7 @@ struct nfsd4_open {
 	stateid_t       op_delegate_stateid; /* request - response */
 	u32		op_create;     	    /* request */
 	u32		op_createmode;      /* request */
-	u32		op_bmval[2];        /* request */
+	u32		op_bmval[3];        /* request */
 	union {                             /* request */
 		struct iattr	iattr;                      /* UNCHECKED4,GUARDED4 */
 		nfs4_verifier	verf;                                /* EXCLUSIVE4 */
@@ -270,7 +270,7 @@ struct nfsd4_readdir {
 	nfs4_verifier	rd_verf;            /* request */
 	u32		rd_dircount;        /* request */
 	u32		rd_maxcount;        /* request */
-	u32		rd_bmval[2];        /* request */
+	u32		rd_bmval[3];        /* request */
 	struct svc_rqst *rd_rqstp;          /* response */
 	struct svc_fh * rd_fhp;             /* response */
 
@@ -312,7 +312,7 @@ struct nfsd4_secinfo {
 
 struct nfsd4_setattr {
 	stateid_t	sa_stateid;         /* request */
-	u32		sa_bmval[2];        /* request */
+	u32		sa_bmval[3];        /* request */
 	struct iattr	sa_iattr;           /* request */
 	struct nfs4_acl *sa_acl;
 };
@@ -338,7 +338,7 @@ struct nfsd4_setclientid_confirm {
 
 /* also used for NVERIFY */
 struct nfsd4_verify {
-	u32		ve_bmval[2];        /* request */
+	u32		ve_bmval[3];        /* request */
 	u32		ve_attrlen;         /* request */
 	char *		ve_attrval;         /* request */
 };
-- 
1.6.2.1

[PATCH v2 45/47] nfsd41: SUPPATTR_EXCLCREAT attribute

[Benny Halevy]

Return bitmask for supported EXCLUSIVE4_1 create attributes.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4xdr.c         |    7 ++++++-
 include/linux/nfs4.h      |    2 ++
 include/linux/nfsd/nfsd.h |   14 +++++++++++++-
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 079626c..288332c 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -2154,7 +2154,12 @@ out_acl:
 		}
 		WRITE64(stat.ino);
 	}
-	BUG_ON(bmval2);	/* FIXME: not implemented yet */
+	if (bmval2 & FATTR4_WORD2_SUPPATTR_EXCLCREAT) {
+		WRITE32(3);
+		WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD0);
+		WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD1);
+		WRITE32(NFSD_SUPPATTR_EXCLCREAT_WORD2);
+	}
 
 	*attrlenp = htonl((char *)p - (char *)attrlenp - 4);
 	*countp = p - buffer;
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index a34683e..735c7d8 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -404,6 +404,8 @@ enum lock_type4 {
 #define FATTR4_WORD0_UNIQUE_HANDLES     (1UL << 9)
 #define FATTR4_WORD0_LEASE_TIME         (1UL << 10)
 #define FATTR4_WORD0_RDATTR_ERROR       (1UL << 11)
+/* Mandatory in NFSv4.1 */
+#define FATTR4_WORD2_SUPPATTR_EXCLCREAT (1UL << 11)
 
 /* Recommended Attributes */
 #define FATTR4_WORD0_ACL                (1UL << 12)
diff --git a/include/linux/nfsd/nfsd.h b/include/linux/nfsd/nfsd.h
index 3dccb76..b9e6682 100644
--- a/include/linux/nfsd/nfsd.h
+++ b/include/linux/nfsd/nfsd.h
@@ -374,7 +374,7 @@ extern struct timeval	nfssvc_boot;
 	NFSD4_SUPPORTED_ATTRS_WORD1
 
 #define NFSD4_1_SUPPORTED_ATTRS_WORD2 \
-	NFSD4_SUPPORTED_ATTRS_WORD2
+	(NFSD4_SUPPORTED_ATTRS_WORD2 | FATTR4_WORD2_SUPPATTR_EXCLCREAT)
 
 #if defined(CONFIG_NFSD_V4_1)
 
@@ -427,6 +427,18 @@ static inline u32 nfsd_suppattrs2(u32 minorversion)
  | FATTR4_WORD1_TIME_ACCESS_SET | FATTR4_WORD1_TIME_MODIFY_SET)
 #define NFSD_WRITEABLE_ATTRS_WORD2 0
 
+#define NFSD_SUPPATTR_EXCLCREAT_WORD0 \
+	NFSD_WRITEABLE_ATTRS_WORD0
+/*
+ * we currently store the exclusive create verifier in the v_{a,m}time
+ * attributes so the client can't set these at create time using EXCLUSIVE4_1
+ */
+#define NFSD_SUPPATTR_EXCLCREAT_WORD1 \
+	(NFSD_WRITEABLE_ATTRS_WORD1 & \
+	 ~(FATTR4_WORD1_TIME_ACCESS_SET | FATTR4_WORD1_TIME_MODIFY_SET))
+#define NFSD_SUPPATTR_EXCLCREAT_WORD2 \
+	NFSD_WRITEABLE_ATTRS_WORD2
+
 #endif /* CONFIG_NFSD_V4 */
 
 #endif /* LINUX_NFSD_NFSD_H */
-- 
1.6.2.1

[PATCH v2 46/47] nfsd41: CREATE_EXCLUSIVE4_1

[Benny Halevy]

Implement the CREATE_EXCLUSIVE4_1 open mode conforming to
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-26

This mode allows the client to atomically create a file
if it doesn't exist while setting some of its attributes.

It must be implemented if the server supports persistent
reply cache and/or pnfs.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c        |   15 +++++++++++++++
 fs/nfsd/nfs4xdr.c         |   17 +++++++++++++++++
 include/linux/nfs4.h      |    8 +++++++-
 include/linux/nfsd/xdr4.h |   10 ++++------
 4 files changed, 43 insertions(+), 7 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f3b560f..6d7d7c5 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -93,6 +93,21 @@ do_open_lookup(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_o
 	open->op_truncate = 0;
 
 	if (open->op_create) {
+		/* FIXME: check session persistence and pnfs flags.
+		 * The nfsv4.1 spec requires the following semantics:
+		 *
+		 * Persistent   | pNFS   | Server REQUIRED | Client Allowed
+		 * Reply Cache  | server |                 |
+		 * -------------+--------+-----------------+--------------------
+		 * no           | no     | EXCLUSIVE4_1    | EXCLUSIVE4_1
+		 *              |        |                 | (SHOULD)
+		 *              |        | and EXCLUSIVE4  | or EXCLUSIVE4
+		 *              |        |                 | (SHOULD NOT)
+		 * no           | yes    | EXCLUSIVE4_1    | EXCLUSIVE4_1
+		 * yes          | no     | GUARDED4        | GUARDED4
+		 * yes          | yes    | GUARDED4        | GUARDED4
+		 */
+
 		/*
 		 * Note: create modes (UNCHECKED,GUARDED...) are the same
 		 * in NFSv4 as in v3.
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 288332c..0084459 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -260,6 +260,12 @@ static u32 nfsd_attrmask[] = {
 	NFSD_WRITEABLE_ATTRS_WORD2
 };
 
+static u32 nfsd41_ex_attrmask[] = {
+	NFSD_SUPPATTR_EXCLCREAT_WORD0,
+	NFSD_SUPPATTR_EXCLCREAT_WORD1,
+	NFSD_SUPPATTR_EXCLCREAT_WORD2
+};
+
 static __be32
 nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval, u32 *writable,
 		   struct iattr *iattr, struct nfs4_acl **acl)
@@ -684,6 +690,17 @@ nfsd4_decode_open(struct nfsd4_compoundargs *argp, struct nfsd4_open *open)
 			READ_BUF(8);
 			COPYMEM(open->op_verf.data, 8);
 			break;
+		case NFS4_CREATE_EXCLUSIVE4_1:
+			if (argp->minorversion < 1)
+				goto xdr_error;
+			READ_BUF(8);
+			COPYMEM(open->op_verf.data, 8);
+			status = nfsd4_decode_fattr(argp, open->op_bmval,
+				nfsd41_ex_attrmask, &open->op_iattr,
+				&open->op_acl);
+			if (status)
+				goto out;
+			break;
 		default:
 			goto xdr_error;
 		}
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 735c7d8..5756359 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -368,7 +368,13 @@ enum opentype4 {
 enum createmode4 {
 	NFS4_CREATE_UNCHECKED = 0,
 	NFS4_CREATE_GUARDED = 1,
-	NFS4_CREATE_EXCLUSIVE = 2
+	NFS4_CREATE_EXCLUSIVE = 2,
+	/*
+	 * New to NFSv4.1. If session is persistent,
+	 * GUARDED4 MUST be used. Otherwise, use
+	 * EXCLUSIVE4_1 instead of EXCLUSIVE4.
+	 */
+	NFS4_CREATE_EXCLUSIVE4_1 = 3
 };
 
 enum limit_by4 {
diff --git a/include/linux/nfsd/xdr4.h b/include/linux/nfsd/xdr4.h
index d7a8050..7ed9955 100644
--- a/include/linux/nfsd/xdr4.h
+++ b/include/linux/nfsd/xdr4.h
@@ -218,10 +218,8 @@ struct nfsd4_open {
 	u32		op_create;     	    /* request */
 	u32		op_createmode;      /* request */
 	u32		op_bmval[3];        /* request */
-	union {                             /* request */
-		struct iattr	iattr;                      /* UNCHECKED4,GUARDED4 */
-		nfs4_verifier	verf;                                /* EXCLUSIVE4 */
-	} u;
+	struct iattr	iattr;              /* UNCHECKED4, GUARDED4, EXCLUSIVE4_1 */
+	nfs4_verifier	verf;               /* EXCLUSIVE4 */
 	clientid_t	op_clientid;        /* request */
 	struct xdr_netobj op_owner;           /* request */
 	u32		op_seqid;           /* request */
@@ -235,8 +233,8 @@ struct nfsd4_open {
 	struct nfs4_stateowner *op_stateowner; /* used during processing */
 	struct nfs4_acl *op_acl;
 };
-#define op_iattr	u.iattr
-#define op_verf		u.verf
+#define op_iattr	iattr
+#define op_verf		verf
 
 struct nfsd4_open_confirm {
 	stateid_t	oc_req_stateid		/* request */;
-- 
1.6.2.1

[PATCH v2 47/47] nfsd41: Documentation/filesystems/nfs41-server.txt

[Benny Halevy]

Initial nfs41 server write up describing the status of the linux
server implementation.

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 Documentation/filesystems/nfs41-server.txt |  148 ++++++++++++++++++++++++++++
 1 files changed, 148 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/filesystems/nfs41-server.txt

diff --git a/Documentation/filesystems/nfs41-server.txt b/Documentation/filesystems/nfs41-server.txt
new file mode 100644
index 0000000..e62cd13
--- /dev/null
+++ b/Documentation/filesystems/nfs41-server.txt
@@ -0,0 +1,148 @@
+NFSv4.1 Server Implementation
+
+The NFSv4 minorversion 1 (NFSv4.1) implementation in nfsd is based
+on the latest NFSv4.1 Internet Draft:
+http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-29
+
+If you're a developer or just eager to test the code, set
+CONFIG_NFSD_V4_1 to 'y' in your kernel configuration file.
+This will allow the server to serve both NFSv4 and NFSv4.1
+clients concurrently.
+
+From the many new features in NFSv4.1 the current implementation
+focuses on the mandatory-to-implement NFSv4.1 Sessions, providing
+"exactly once" semantics and better control and throttling of the
+resources allocated for each client.
+
+Other NFSv4.1 features, Parallel NFS operations in particular,
+are still under development out of tree.
+See http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design
+for more information.
+
+The table below, taken from the NFSv4.1 document, lists
+the operations that are mandatory to implement (REQ), optional
+(OPT), and NFSv4.0 operations that are required not to implement (MNI)
+in minor version 1.  The first column indicates the operations that
+are not supported yet by the linux server implementation.
+
+The OPTIONAL features identified and their abbreviations are as follows:
+	pNFS	Parallel NFS
+	FDELG	File Delegations
+	DDELG	Directory Delegations
+
+The following abbreviations indicate the linux server implementation status.
+	I	Implemented NFSv4.1 operations.
+	NS	Not Supported.
+	NS*	unimplemented optional feature.
+	P	pNFS features implemented out of tree.
+	PNS	pNFS features that are not supported yet (out of tree).
+
+Operations
+
+   +----------------------+------------+--------------+----------------+
+   | Operation            | REQ, REC,  | Feature      | Definition     |
+   |                      | OPT, or    | (REQ, REC,   |                |
+   |                      | MNI        | or OPT)      |                |
+   +----------------------+------------+--------------+----------------+
+   | ACCESS               | REQ        |              | Section 18.1   |
+NS | BACKCHANNEL_CTL      | REQ        |              | Section 18.33  |
+NS | BIND_CONN_TO_SESSION | REQ        |              | Section 18.34  |
+   | CLOSE                | REQ        |              | Section 18.2   |
+   | COMMIT               | REQ        |              | Section 18.3   |
+   | CREATE               | REQ        |              | Section 18.4   |
+I  | CREATE_SESSION       | REQ        |              | Section 18.36  |
+NS*| DELEGPURGE           | OPT        | FDELG (REQ)  | Section 18.5   |
+   | DELEGRETURN          | OPT        | FDELG,       | Section 18.6   |
+   |                      |            | DDELG, pNFS  |                |
+   |                      |            | (REQ)        |                |
+NS | DESTROY_CLIENTID     | REQ        |              | Section 18.50  |
+I  | DESTROY_SESSION      | REQ        |              | Section 18.37  |
+I  | EXCHANGE_ID          | REQ        |              | Section 18.35  |
+NS | FREE_STATEID         | REQ        |              | Section 18.38  |
+   | GETATTR              | REQ        |              | Section 18.7   |
+P  | GETDEVICEINFO        | OPT        | pNFS (REQ)   | Section 18.40  |
+P  | GETDEVICELIST        | OPT        | pNFS (OPT)   | Section 18.41  |
+   | GETFH                | REQ        |              | Section 18.8   |
+NS*| GET_DIR_DELEGATION   | OPT        | DDELG (REQ)  | Section 18.39  |
+P  | LAYOUTCOMMIT         | OPT        | pNFS (REQ)   | Section 18.42  |
+P  | LAYOUTGET            | OPT        | pNFS (REQ)   | Section 18.43  |
+P  | LAYOUTRETURN         | OPT        | pNFS (REQ)   | Section 18.44  |
+   | LINK                 | OPT        |              | Section 18.9   |
+   | LOCK                 | REQ        |              | Section 18.10  |
+   | LOCKT                | REQ        |              | Section 18.11  |
+   | LOCKU                | REQ        |              | Section 18.12  |
+   | LOOKUP               | REQ        |              | Section 18.13  |
+   | LOOKUPP              | REQ        |              | Section 18.14  |
+   | NVERIFY              | REQ        |              | Section 18.15  |
+   | OPEN                 | REQ        |              | Section 18.16  |
+NS*| OPENATTR             | OPT        |              | Section 18.17  |
+   | OPEN_CONFIRM         | MNI        |              | N/A            |
+   | OPEN_DOWNGRADE       | REQ        |              | Section 18.18  |
+   | PUTFH                | REQ        |              | Section 18.19  |
+   | PUTPUBFH             | REQ        |              | Section 18.20  |
+   | PUTROOTFH            | REQ        |              | Section 18.21  |
+   | READ                 | REQ        |              | Section 18.22  |
+   | READDIR              | REQ        |              | Section 18.23  |
+   | READLINK             | OPT        |              | Section 18.24  |
+NS | RECLAIM_COMPLETE     | REQ        |              | Section 18.51  |
+   | RELEASE_LOCKOWNER    | MNI        |              | N/A            |
+   | REMOVE               | REQ        |              | Section 18.25  |
+   | RENAME               | REQ        |              | Section 18.26  |
+   | RENEW                | MNI        |              | N/A            |
+   | RESTOREFH            | REQ        |              | Section 18.27  |
+   | SAVEFH               | REQ        |              | Section 18.28  |
+   | SECINFO              | REQ        |              | Section 18.29  |
+NS | SECINFO_NO_NAME      | REC        | pNFS files   | Section 18.45, |
+   |                      |            | layout (REQ) | Section 13.12  |
+I  | SEQUENCE             | REQ        |              | Section 18.46  |
+   | SETATTR              | REQ        |              | Section 18.30  |
+   | SETCLIENTID          | MNI        |              | N/A            |
+   | SETCLIENTID_CONFIRM  | MNI        |              | N/A            |
+NS | SET_SSV              | REQ        |              | Section 18.47  |
+NS | TEST_STATEID         | REQ        |              | Section 18.48  |
+   | VERIFY               | REQ        |              | Section 18.31  |
+NS*| WANT_DELEGATION      | OPT        | FDELG (OPT)  | Section 18.49  |
+   | WRITE                | REQ        |              | Section 18.32  |
+
+Callback Operations
+
+   +-------------------------+-----------+-------------+---------------+
+   | Operation               | REQ, REC, | Feature     | Definition    |
+   |                         | OPT, or   | (REQ, REC,  |               |
+   |                         | MNI       | or OPT)     |               |
+   +-------------------------+-----------+-------------+---------------+
+   | CB_GETATTR              | OPT       | FDELG (REQ) | Section 20.1  |
+P  | CB_LAYOUTRECALL         | OPT       | pNFS (REQ)  | Section 20.3  |
+NS*| CB_NOTIFY               | OPT       | DDELG (REQ) | Section 20.4  |
+P  | CB_NOTIFY_DEVICEID      | OPT       | pNFS (OPT)  | Section 20.12 |
+NS*| CB_NOTIFY_LOCK          | OPT       |             | Section 20.11 |
+NS*| CB_PUSH_DELEG           | OPT       | FDELG (OPT) | Section 20.5  |
+   | CB_RECALL               | OPT       | FDELG,      | Section 20.2  |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+NS*| CB_RECALL_ANY           | OPT       | FDELG,      | Section 20.6  |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+NS | CB_RECALL_SLOT          | REQ       |             | Section 20.8  |
+NS*| CB_RECALLABLE_OBJ_AVAIL | OPT       | DDELG, pNFS | Section 20.7  |
+   |                         |           | (REQ)       |               |
+I  | CB_SEQUENCE             | OPT       | FDELG,      | Section 20.9  |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+NS*| CB_WANTS_CANCELLED      | OPT       | FDELG,      | Section 20.10 |
+   |                         |           | DDELG, pNFS |               |
+   |                         |           | (REQ)       |               |
+   +-------------------------+-----------+-------------+---------------+
+
+Implementation notes:
+
+EXCHANGE_ID:
+* only SP4_NONE state protection supported
+* implementation ids are ignored
+
+CREATE_SESSION:
+* backchannel attributes are ignored
+* backchannel security parameters are ignored
+
+SEQUENCE:
+* no support for dynamic slot table renegotiation (optional)
-- 
1.6.2.1

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> >> Hi Bruce,
> >>
> >> Here's the latest server patches implementing the NFSv4.1
> >> Sessions features.
> >>
> >> This patchset is based over your for-2.6.30 branch
> >> and is also available from
> >> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> > 
> > There's a few merge conflicts with my current for-2.6.30--probably my
> > fault for not pushing that out recently enough, apologies.  Would you
> > mind updating?
> > 
> > --b.
> > 
> 
> Sure. Here's a rebased version with two minor changes (see below)

Got it, thanks!  Reading....

--b.

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> >> Hi Bruce,
> >>
> >> Here's the latest server patches implementing the NFSv4.1
> >> Sessions features.
> >>
> >> This patchset is based over your for-2.6.30 branch
> >> and is also available from
> >> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> > 
> > There's a few merge conflicts with my current for-2.6.30--probably my
> > fault for not pushing that out recently enough, apologies.  Would you
> > mind updating?
> > 
> > --b.
> > 
> 
> Sure. Here's a rebased version with two minor changes (see below)

On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
dereference in the callback code.  Looks like it probably happened while
running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so
far.

--b.

BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200
*pde = 00000000 
Oops: 0000 [#1] PREEMPT 
last sysfs file: /sys/kernel/uevent_seqnum
Modules linked in:

Pid: 3949, comm: nfs4_cb_recall Not tainted (2.6.29-rc8-00312-ga564667 #39) 
EIP: 0060:[<c03e4e1e>] EFLAGS: 00010286 CPU: 0
EIP is at nfs4_xdr_dec_cb_recall+0x4e/0x200
EAX: c7bb22a0 EBX: c7bb2298 ECX: c7bb22a0 EDX: c7bb22a4
ESI: 00000000 EDI: c66e3000 EBP: c5281ea4 ESP: c5281e6c
 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
Process nfs4_cb_recall (pid: 3949, ti=c5280000 task=c66a6b60 task.ti=c5280000)
Stack:
 c66a6b60 00000000 c5281eb4 c086cab4 c0aa0380 00000001 c11bd4a4 c7bb22a4
 c66e302c c7bb22ac c66e302c 00000000 c6cf14e8 c66e3000 c5281ecc c08199c3
 c0819aae 00000000 c5281ee8 c5281ef0 c03e4dd0 c6cf14e8 c7bb2298 c03e4dd0
Call Trace:
 [<c086cab4>] ? schedule+0x354/0x540
 [<c08199c3>] ? rpcauth_unwrap_resp+0x63/0x90
 [<c0819aae>] ? rpcauth_checkverf+0x2e/0x70
 [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
 [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
 [<c08122ce>] ? call_decode+0x1ae/0x820
 [<c086d125>] ? out_of_line_wait_on_bit+0x65/0x80
 [<c0818720>] ? rpc_wait_bit_killable+0x0/0x40
 [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
 [<c0818e62>] ? __rpc_execute+0x92/0x290
 [<c086f4fc>] ? _spin_unlock+0x2c/0x50
 [<c08186c7>] ? rpc_set_active+0x67/0x80
 [<c081907e>] ? rpc_execute+0x1e/0x30
 [<c08129e5>] ? rpc_run_task+0x35/0x70
 [<c0812b40>] ? rpc_call_sync+0x40/0x70
 [<c03e51e0>] ? nfsd4_cb_recall+0x70/0x130
 [<c086cab4>] ? schedule+0x354/0x540
 [<c0247d1b>] ? trace_hardirqs_on+0xb/0x10
 [<c03dfd40>] ? do_recall+0x0/0x20
 [<c03dfd54>] ? do_recall+0x14/0x20
 [<c023941f>] ? kthread+0x3f/0x70
 [<c02393e0>] ? kthread+0x0/0x70
 [<c0203b67>] ? kernel_thread_helper+0x7/0x10
Code: 43 00 ba 08 00 00 00 8d 45 e4 e8 0e c1 43 00 85 c0 74 3a 8b 50 04 8d 45 e4 0f ca 83 c2 04 e8 fa c0 43 00 85 c0 0f 84 8a 00 00 00 <8b> 06 8b 00 85 c0 75 32 ba 04 00 00 00 8d 45 e4 e8 cd fe ff ff 
EIP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200 SS:ESP 0068:c5281e6c
---[ end trace 2724475d9856cb6c ]---

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Mon, Mar 30, 2009 at 02:33:38PM -0400, bfields wrote:
> On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
> > On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > > On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> > >> Hi Bruce,
> > >>
> > >> Here's the latest server patches implementing the NFSv4.1
> > >> Sessions features.
> > >>
> > >> This patchset is based over your for-2.6.30 branch
> > >> and is also available from
> > >> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> > > 
> > > There's a few merge conflicts with my current for-2.6.30--probably my
> > > fault for not pushing that out recently enough, apologies.  Would you
> > > mind updating?
> > > 
> > > --b.
> > > 
> > 
> > Sure. Here's a rebased version with two minor changes (see below)
> 
> On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
> dereference in the callback code.  Looks like it probably happened while
> running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so
> far.

Nevermind about the krb5p, that can't be right.  Perhaps
cbs_minorversion just isn't getting set correctly, and it's trying to
access an uninitialized cbs_clp?

--b.

> 
> --b.
> 
> BUG: unable to handle kernel NULL pointer dereference at (null)
> IP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200
> *pde = 00000000 
> Oops: 0000 [#1] PREEMPT 
> last sysfs file: /sys/kernel/uevent_seqnum
> Modules linked in:
> 
> Pid: 3949, comm: nfs4_cb_recall Not tainted (2.6.29-rc8-00312-ga564667 #39) 
> EIP: 0060:[<c03e4e1e>] EFLAGS: 00010286 CPU: 0
> EIP is at nfs4_xdr_dec_cb_recall+0x4e/0x200
> EAX: c7bb22a0 EBX: c7bb2298 ECX: c7bb22a0 EDX: c7bb22a4
> ESI: 00000000 EDI: c66e3000 EBP: c5281ea4 ESP: c5281e6c
>  DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
> Process nfs4_cb_recall (pid: 3949, ti=c5280000 task=c66a6b60 task.ti=c5280000)
> Stack:
>  c66a6b60 00000000 c5281eb4 c086cab4 c0aa0380 00000001 c11bd4a4 c7bb22a4
>  c66e302c c7bb22ac c66e302c 00000000 c6cf14e8 c66e3000 c5281ecc c08199c3
>  c0819aae 00000000 c5281ee8 c5281ef0 c03e4dd0 c6cf14e8 c7bb2298 c03e4dd0
> Call Trace:
>  [<c086cab4>] ? schedule+0x354/0x540
>  [<c08199c3>] ? rpcauth_unwrap_resp+0x63/0x90
>  [<c0819aae>] ? rpcauth_checkverf+0x2e/0x70
>  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
>  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
>  [<c08122ce>] ? call_decode+0x1ae/0x820
>  [<c086d125>] ? out_of_line_wait_on_bit+0x65/0x80
>  [<c0818720>] ? rpc_wait_bit_killable+0x0/0x40
>  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
>  [<c0818e62>] ? __rpc_execute+0x92/0x290
>  [<c086f4fc>] ? _spin_unlock+0x2c/0x50
>  [<c08186c7>] ? rpc_set_active+0x67/0x80
>  [<c081907e>] ? rpc_execute+0x1e/0x30
>  [<c08129e5>] ? rpc_run_task+0x35/0x70
>  [<c0812b40>] ? rpc_call_sync+0x40/0x70
>  [<c03e51e0>] ? nfsd4_cb_recall+0x70/0x130
>  [<c086cab4>] ? schedule+0x354/0x540
>  [<c0247d1b>] ? trace_hardirqs_on+0xb/0x10
>  [<c03dfd40>] ? do_recall+0x0/0x20
>  [<c03dfd54>] ? do_recall+0x14/0x20
>  [<c023941f>] ? kthread+0x3f/0x70
>  [<c02393e0>] ? kthread+0x0/0x70
>  [<c0203b67>] ? kernel_thread_helper+0x7/0x10
> Code: 43 00 ba 08 00 00 00 8d 45 e4 e8 0e c1 43 00 85 c0 74 3a 8b 50 04 8d 45 e4 0f ca 83 c2 04 e8 fa c0 43 00 85 c0 0f 84 8a 00 00 00 <8b> 06 8b 00 85 c0 75 32 ba 04 00 00 00 8d 45 e4 e8 cd fe ff ff 
> EIP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200 SS:ESP 0068:c5281e6c
> ---[ end trace 2724475d9856cb6c ]---
> 

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Benny Halevy]

On Mar. 30, 2009, 21:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
>> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
>>>> Hi Bruce,
>>>>
>>>> Here's the latest server patches implementing the NFSv4.1
>>>> Sessions features.
>>>>
>>>> This patchset is based over your for-2.6.30 branch
>>>> and is also available from
>>>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
>>> There's a few merge conflicts with my current for-2.6.30--probably my
>>> fault for not pushing that out recently enough, apologies.  Would you
>>> mind updating?
>>>
>>> --b.
>>>
>> Sure. Here's a rebased version with two minor changes (see below)
> 
> On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
> dereference in the callback code.  Looks like it probably happened while
> running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so

Weird. nfs4_xdr_dec_cb_recall+0x4e doesn't seem like a valid IP
for nfsd.ko @a564667.  Bruce, can you please send me you .config file?

We know the current implementation is still incorrect for krb5,
but I don't think it's supposed to crash either.
Ricardo, have you run into this by any chance?

Benny

> far.
> 
> --b.
> 
> BUG: unable to handle kernel NULL pointer dereference at (null)
> IP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200
> *pde = 00000000 
> Oops: 0000 [#1] PREEMPT 
> last sysfs file: /sys/kernel/uevent_seqnum
> Modules linked in:
> 
> Pid: 3949, comm: nfs4_cb_recall Not tainted (2.6.29-rc8-00312-ga564667 #39) 
> EIP: 0060:[<c03e4e1e>] EFLAGS: 00010286 CPU: 0
> EIP is at nfs4_xdr_dec_cb_recall+0x4e/0x200
> EAX: c7bb22a0 EBX: c7bb2298 ECX: c7bb22a0 EDX: c7bb22a4
> ESI: 00000000 EDI: c66e3000 EBP: c5281ea4 ESP: c5281e6c
>  DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
> Process nfs4_cb_recall (pid: 3949, ti=c5280000 task=c66a6b60 task.ti=c5280000)
> Stack:
>  c66a6b60 00000000 c5281eb4 c086cab4 c0aa0380 00000001 c11bd4a4 c7bb22a4
>  c66e302c c7bb22ac c66e302c 00000000 c6cf14e8 c66e3000 c5281ecc c08199c3
>  c0819aae 00000000 c5281ee8 c5281ef0 c03e4dd0 c6cf14e8 c7bb2298 c03e4dd0
> Call Trace:
>  [<c086cab4>] ? schedule+0x354/0x540
>  [<c08199c3>] ? rpcauth_unwrap_resp+0x63/0x90
>  [<c0819aae>] ? rpcauth_checkverf+0x2e/0x70
>  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
>  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
>  [<c08122ce>] ? call_decode+0x1ae/0x820
>  [<c086d125>] ? out_of_line_wait_on_bit+0x65/0x80
>  [<c0818720>] ? rpc_wait_bit_killable+0x0/0x40
>  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
>  [<c0818e62>] ? __rpc_execute+0x92/0x290
>  [<c086f4fc>] ? _spin_unlock+0x2c/0x50
>  [<c08186c7>] ? rpc_set_active+0x67/0x80
>  [<c081907e>] ? rpc_execute+0x1e/0x30
>  [<c08129e5>] ? rpc_run_task+0x35/0x70
>  [<c0812b40>] ? rpc_call_sync+0x40/0x70
>  [<c03e51e0>] ? nfsd4_cb_recall+0x70/0x130
>  [<c086cab4>] ? schedule+0x354/0x540
>  [<c0247d1b>] ? trace_hardirqs_on+0xb/0x10
>  [<c03dfd40>] ? do_recall+0x0/0x20
>  [<c03dfd54>] ? do_recall+0x14/0x20
>  [<c023941f>] ? kthread+0x3f/0x70
>  [<c02393e0>] ? kthread+0x0/0x70
>  [<c0203b67>] ? kernel_thread_helper+0x7/0x10
> Code: 43 00 ba 08 00 00 00 8d 45 e4 e8 0e c1 43 00 85 c0 74 3a 8b 50 04 8d 45 e4 0f ca 83 c2 04 e8 fa c0 43 00 85 c0 0f 84 8a 00 00 00 <8b> 06 8b 00 85 c0 75 32 ba 04 00 00 00 8d 45 e4 e8 cd fe ff ff 
> EIP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200 SS:ESP 0068:c5281e6c
> ---[ end trace 2724475d9856cb6c ]---
> 

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Mon, Mar 30, 2009 at 10:03:39PM +0300, Benny Halevy wrote:
> 
> On Mar. 30, 2009, 21:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
> >> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> >>>> Hi Bruce,
> >>>>
> >>>> Here's the latest server patches implementing the NFSv4.1
> >>>> Sessions features.
> >>>>
> >>>> This patchset is based over your for-2.6.30 branch
> >>>> and is also available from
> >>>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> >>> There's a few merge conflicts with my current for-2.6.30--probably my
> >>> fault for not pushing that out recently enough, apologies.  Would you
> >>> mind updating?
> >>>
> >>> --b.
> >>>
> >> Sure. Here's a rebased version with two minor changes (see below)
> > 
> > On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
> > dereference in the callback code.  Looks like it probably happened while
> > running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so
> 
> Weird. nfs4_xdr_dec_cb_recall+0x4e doesn't seem like a valid IP
> for nfsd.ko @a564667.  Bruce, can you please send me you .config file?

I'm building without CONFIG_NFSD_V4_1 for now; .config appended.

Start of my nfs4_xdr_dec_cb_recall, for what it's worth:

c03e4dd0 <nfs4_xdr_dec_cb_recall>:
c03e4dd0:       55                      push   %ebp
c03e4dd1:       89 e5                   mov    %esp,%ebp
c03e4dd3:       83 ec 38                sub    $0x38,%esp
c03e4dd6:       89 5d f4                mov    %ebx,-0xc(%ebp)
c03e4dd9:       89 75 f8                mov    %esi,-0x8(%ebp)
c03e4ddc:       89 7d fc                mov    %edi,-0x4(%ebp)
c03e4ddf:       e8 8c ed e1 ff          call   c0203b70 <mcount>
c03e4de4:       89 d3                   mov    %edx,%ebx
c03e4de6:       89 ce                   mov    %ecx,%esi
c03e4de8:       8d 50 2c                lea    0x2c(%eax),%edx
c03e4deb:       89 d9                   mov    %ebx,%ecx
c03e4ded:       8d 45 e4                lea    -0x1c(%ebp),%eax
c03e4df0:       e8 eb c0 43 00          call   c0820ee0 <xdr_init_decode>
c03e4df5:       ba 08 00 00 00          mov    $0x8,%edx
c03e4dfa:       8d 45 e4                lea    -0x1c(%ebp),%eax
c03e4dfd:       e8 0e c1 43 00          call   c0820f10 <xdr_inline_decode>
c03e4e02:       85 c0                   test   %eax,%eax
c03e4e04:       74 3a                   je     c03e4e40 <nfs4_xdr_dec_cb_recall+0x70>
c03e4e06:       8b 50 04                mov    0x4(%eax),%edx
c03e4e09:       8d 45 e4                lea    -0x1c(%ebp),%eax
c03e4e0c:       0f ca                   bswap  %edx
c03e4e0e:       83 c2 04                add    $0x4,%edx
c03e4e11:       e8 fa c0 43 00          call   c0820f10 <xdr_inline_decode>
c03e4e16:       85 c0                   test   %eax,%eax
c03e4e18:       0f 84 8a 00 00 00       je     c03e4ea8 <nfs4_xdr_dec_cb_recall+0xd8>
--> c03e4e1e:   8b 06                   mov    (%esi),%eax
c03e4e20:       8b 00                   mov    (%eax),%eax
c03e4e22:       85 c0                   test   %eax,%eax
c03e4e24:       75 32                   jne    c03e4e58 <nfs4_xdr_dec_cb_recall+0x88>
c03e4e26:       ba 04 00 00 00          mov    $0x4,%edx
c03e4e2b:       8d 45 e4                lea    -0x1c(%ebp),%eax
c03e4e2e:       e8 cd fe ff ff          call   c03e4d00 <decode_cb_op_hdr>
c03e4e33:       8b 5d f4                mov    -0xc(%ebp),%ebx
c03e4e36:       8b 75 f8                mov    -0x8(%ebp),%esi
c03e4e39:       8b 7d fc                mov    -0x4(%ebp),%edi
c03e4e3c:       89 ec                   mov    %ebp,%esp
c03e4e3e:       5d                      pop    %ebp
c03e4e3f:       c3                      ret
...


--b.

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.29-rc8
# Mon Mar 30 14:07:10 2009
#
# CONFIG_64BIT is not set
CONFIG_X86_32=y
# CONFIG_X86_64 is not set
CONFIG_X86=y
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_FAST_CMPXCHG_LOCAL=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
# CONFIG_GENERIC_TIME_VSYSCALL is not set
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
# CONFIG_HAVE_SETUP_PER_CPU_AREA is not set
# CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
# CONFIG_ZONE_DMA32 is not set
CONFIG_ARCH_POPULATES_NODE_MAP=y
# CONFIG_AUDIT_ARCH is not set
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_KTIME_SCALAR=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_BROKEN_ON_SMP=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
# CONFIG_TASKSTATS is not set
# CONFIG_AUDIT is not set

#
# RCU Subsystem
#
CONFIG_CLASSIC_RCU=y
# CONFIG_TREE_RCU is not set
# CONFIG_PREEMPT_RCU is not set
# CONFIG_TREE_RCU_TRACE is not set
# CONFIG_PREEMPT_RCU_TRACE is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=20
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_GROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
# CONFIG_RT_GROUP_SCHED is not set
CONFIG_USER_SCHED=y
# CONFIG_CGROUP_SCHED is not set
# CONFIG_CGROUPS is not set
CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y
CONFIG_RELAY=y
CONFIG_NAMESPACES=y
# CONFIG_UTS_NS is not set
# CONFIG_IPC_NS is not set
# CONFIG_USER_NS is not set
# CONFIG_PID_NS is not set
# CONFIG_NET_NS is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
# CONFIG_EMBEDDED is not set
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
CONFIG_COMPAT_BRK=y
CONFIG_SLAB=y
# CONFIG_SLUB is not set
# CONFIG_SLOB is not set
CONFIG_PROFILING=y
CONFIG_TRACEPOINTS=y
CONFIG_MARKERS=y
CONFIG_OPROFILE=y
CONFIG_HAVE_OPROFILE=y
CONFIG_KPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_GENERIC_DMA_COHERENT=y
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_BLOCK=y
CONFIG_LBD=y
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_BLK_DEV_BSG is not set
# CONFIG_BLK_DEV_INTEGRITY is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_AS=y
# CONFIG_DEFAULT_DEADLINE is not set
# CONFIG_DEFAULT_CFQ is not set
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="anticipatory"
CONFIG_FREEZER=y

#
# Processor type and features
#
# CONFIG_NO_HZ is not set
# CONFIG_HIGH_RES_TIMERS is not set
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
# CONFIG_SMP is not set
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_VSMP is not set
# CONFIG_X86_RDC321X is not set
CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
# CONFIG_MEMTEST is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
CONFIG_M686=y
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_GENERIC_CPU is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CPU=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=5
CONFIG_X86_XADD=y
# CONFIG_X86_PPRO_FENCE is not set
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=4
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_CYRIX_32=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_CENTAUR_32=y
CONFIG_CPU_SUP_TRANSMETA_32=y
CONFIG_CPU_SUP_UMC_32=y
CONFIG_X86_DS=y
CONFIG_X86_PTRACE_BTS=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
# CONFIG_IOMMU_HELPER is not set
# CONFIG_IOMMU_API is not set
CONFIG_NR_CPUS=1
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
# CONFIG_X86_UP_APIC is not set
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_NONFATAL is not set
CONFIG_VM86=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_X86_REBOOTFIXUPS is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_HIGHMEM=y
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
CONFIG_NEED_NODE_MEMMAP_SIZE=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_SELECT_MEMORY_MODEL=y
# CONFIG_FLATMEM_MANUAL is not set
# CONFIG_DISCONTIGMEM_MANUAL is not set
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_STATIC=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_PHYS_ADDR_T_64BIT is not set
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_UNEVICTABLE_LRU=y
# CONFIG_HIGHPTE is not set
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
CONFIG_X86_RESERVE_LOW_64K=y
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_MTRR_SANITIZER is not set
# CONFIG_X86_PAT is not set
# CONFIG_EFI is not set
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
# CONFIG_SCHED_HRTICK is not set
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x100000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x200000
CONFIG_COMPAT_VDSO=y
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management and ACPI options
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
CONFIG_PM_SLEEP=y
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
# CONFIG_HIBERNATION is not set
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_PROCFS_POWER=y
CONFIG_ACPI_SYSFS_POWER=y
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set
# CONFIG_ACPI_SBS is not set
# CONFIG_APM is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
# CONFIG_CPU_FREQ_DEBUG is not set
CONFIG_CPU_FREQ_STAT=y
# CONFIG_CPU_FREQ_STAT_DETAILS is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_GOV_ONDEMAND is not set
# CONFIG_CPU_FREQ_GOV_CONSERVATIVE is not set

#
# CPUFreq processor drivers
#
# CONFIG_X86_ACPI_CPUFREQ is not set
# CONFIG_X86_POWERNOW_K6 is not set
# CONFIG_X86_POWERNOW_K7 is not set
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_GX_SUSPMOD is not set
CONFIG_X86_SPEEDSTEP_CENTRINO=y
CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
# CONFIG_X86_SPEEDSTEP_ICH is not set
# CONFIG_X86_SPEEDSTEP_SMI is not set
# CONFIG_X86_P4_CLOCKMOD is not set
# CONFIG_X86_CPUFREQ_NFORCE2 is not set
# CONFIG_X86_LONGRUN is not set
# CONFIG_X86_LONGHAUL is not set
# CONFIG_X86_E_POWERSAVER is not set

#
# shared options
#
# CONFIG_X86_SPEEDSTEP_LIB is not set
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
# CONFIG_PCI_GOOLPC is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
# CONFIG_PCIEPORTBUS is not set
# CONFIG_ARCH_SUPPORTS_MSI is not set
CONFIG_PCI_LEGACY=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
CONFIG_ISA_DMA_API=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
# CONFIG_OLPC is not set
CONFIG_K8_NB=y
CONFIG_PCCARD=y
# CONFIG_PCMCIA_DEBUG is not set
CONFIG_PCMCIA=y
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_PCMCIA_IOCTL=y
CONFIG_CARDBUS=y

#
# PC-card bridges
#
CONFIG_YENTA=y
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
# CONFIG_PD6729 is not set
# CONFIG_I82092 is not set
CONFIG_PCCARD_NONSTATIC=y
CONFIG_HOTPLUG_PCI=y
# CONFIG_HOTPLUG_PCI_FAKE is not set
# CONFIG_HOTPLUG_PCI_COMPAQ is not set
# CONFIG_HOTPLUG_PCI_ACPI is not set
# CONFIG_HOTPLUG_PCI_CPCI is not set
CONFIG_HOTPLUG_PCI_SHPC=y

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
CONFIG_HAVE_AOUT=y
# CONFIG_BINFMT_AOUT is not set
CONFIG_BINFMT_MISC=y
CONFIG_HAVE_ATOMIC_IOMAP=y
CONFIG_NET=y

#
# Networking options
#
CONFIG_COMPAT_NET_DEV_OPS=y
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_XFRM=y
# CONFIG_XFRM_USER is not set
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
# CONFIG_XFRM_STATISTICS is not set
CONFIG_NET_KEY=y
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
CONFIG_INET_TUNNEL=y
CONFIG_INET_XFRM_MODE_TRANSPORT=y
CONFIG_INET_XFRM_MODE_TUNNEL=y
CONFIG_INET_XFRM_MODE_BEET=y
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
CONFIG_IPV6=y
# CONFIG_IPV6_PRIVACY is not set
# CONFIG_IPV6_ROUTER_PREF is not set
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
# CONFIG_INET6_AH is not set
# CONFIG_INET6_ESP is not set
# CONFIG_INET6_IPCOMP is not set
# CONFIG_IPV6_MIP6 is not set
# CONFIG_INET6_XFRM_TUNNEL is not set
# CONFIG_INET6_TUNNEL is not set
CONFIG_INET6_XFRM_MODE_TRANSPORT=y
CONFIG_INET6_XFRM_MODE_TUNNEL=y
CONFIG_INET6_XFRM_MODE_BEET=y
# CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
CONFIG_IPV6_SIT=y
CONFIG_IPV6_NDISC_NODETYPE=y
# CONFIG_IPV6_TUNNEL is not set
# CONFIG_IPV6_MULTIPLE_TABLES is not set
# CONFIG_IPV6_MROUTE is not set
# CONFIG_NETLABEL is not set
# CONFIG_NETWORK_SECMARK is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_NETFILTER_ADVANCED=y

#
# Core Netfilter Configuration
#
# CONFIG_NETFILTER_NETLINK_QUEUE is not set
# CONFIG_NETFILTER_NETLINK_LOG is not set
# CONFIG_NF_CONNTRACK is not set
CONFIG_NETFILTER_XTABLES=y
# CONFIG_NETFILTER_XT_TARGET_CLASSIFY is not set
# CONFIG_NETFILTER_XT_TARGET_MARK is not set
# CONFIG_NETFILTER_XT_TARGET_NFLOG is not set
# CONFIG_NETFILTER_XT_TARGET_NFQUEUE is not set
# CONFIG_NETFILTER_XT_TARGET_RATEEST is not set
# CONFIG_NETFILTER_XT_TARGET_TCPMSS is not set
# CONFIG_NETFILTER_XT_MATCH_COMMENT is not set
# CONFIG_NETFILTER_XT_MATCH_DCCP is not set
# CONFIG_NETFILTER_XT_MATCH_DSCP is not set
# CONFIG_NETFILTER_XT_MATCH_ESP is not set
# CONFIG_NETFILTER_XT_MATCH_HASHLIMIT is not set
# CONFIG_NETFILTER_XT_MATCH_IPRANGE is not set
# CONFIG_NETFILTER_XT_MATCH_LENGTH is not set
# CONFIG_NETFILTER_XT_MATCH_LIMIT is not set
# CONFIG_NETFILTER_XT_MATCH_MAC is not set
# CONFIG_NETFILTER_XT_MATCH_MARK is not set
# CONFIG_NETFILTER_XT_MATCH_MULTIPORT is not set
# CONFIG_NETFILTER_XT_MATCH_OWNER is not set
# CONFIG_NETFILTER_XT_MATCH_POLICY is not set
# CONFIG_NETFILTER_XT_MATCH_PKTTYPE is not set
# CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
# CONFIG_NETFILTER_XT_MATCH_RATEEST is not set
# CONFIG_NETFILTER_XT_MATCH_REALM is not set
# CONFIG_NETFILTER_XT_MATCH_RECENT is not set
# CONFIG_NETFILTER_XT_MATCH_SCTP is not set
# CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
# CONFIG_NETFILTER_XT_MATCH_STRING is not set
# CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
# CONFIG_NETFILTER_XT_MATCH_TIME is not set
# CONFIG_NETFILTER_XT_MATCH_U32 is not set
# CONFIG_IP_VS is not set

#
# IP: Netfilter Configuration
#
# CONFIG_NF_DEFRAG_IPV4 is not set
CONFIG_IP_NF_QUEUE=y
CONFIG_IP_NF_IPTABLES=y
# CONFIG_IP_NF_MATCH_ADDRTYPE is not set
# CONFIG_IP_NF_MATCH_AH is not set
# CONFIG_IP_NF_MATCH_ECN is not set
# CONFIG_IP_NF_MATCH_TTL is not set
# CONFIG_IP_NF_FILTER is not set
# CONFIG_IP_NF_TARGET_LOG is not set
# CONFIG_IP_NF_TARGET_ULOG is not set
# CONFIG_IP_NF_MANGLE is not set
# CONFIG_IP_NF_RAW is not set
# CONFIG_IP_NF_SECURITY is not set
# CONFIG_IP_NF_ARPTABLES is not set

#
# IPv6: Netfilter Configuration
#
# CONFIG_IP6_NF_QUEUE is not set
# CONFIG_IP6_NF_IPTABLES is not set
# CONFIG_IP_DCCP is not set
CONFIG_IP_SCTP=y
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_NET_DSA is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_SCHED is not set
# CONFIG_DCB is not set

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NET_TCPPROBE is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
# CONFIG_PHONET is not set
CONFIG_WIRELESS=y
# CONFIG_CFG80211 is not set
CONFIG_WIRELESS_OLD_REGULATORY=y
CONFIG_WIRELESS_EXT=y
CONFIG_WIRELESS_EXT_SYSFS=y
# CONFIG_LIB80211 is not set
# CONFIG_MAC80211 is not set
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
CONFIG_PARPORT=y
CONFIG_PARPORT_PC=y
# CONFIG_PARPORT_SERIAL is not set
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_PC_SUPERIO=y
# CONFIG_PARPORT_PC_PCMCIA is not set
# CONFIG_PARPORT_GSC is not set
# CONFIG_PARPORT_AX88796 is not set
CONFIG_PARPORT_1284=y
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_FD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
# CONFIG_BLK_DEV_RAM is not set
# CONFIG_CDROM_PKTCDVD is not set
CONFIG_ATA_OVER_ETH=y
# CONFIG_VIRTIO_BLK is not set
# CONFIG_BLK_DEV_HD is not set
CONFIG_MISC_DEVICES=y
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
# CONFIG_SGI_IOC4 is not set
# CONFIG_TIFM_CORE is not set
# CONFIG_ICS932S401 is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_HP_ILO is not set
# CONFIG_C2PORT is not set

#
# EEPROM support
#
# CONFIG_EEPROM_AT24 is not set
# CONFIG_EEPROM_LEGACY is not set
# CONFIG_EEPROM_93CX6 is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
# CONFIG_SCSI_SCAN_ASYNC is not set
CONFIG_SCSI_WAIT_SCAN=m

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
CONFIG_SCSI_LOWLEVEL=y
# CONFIG_ISCSI_TCP is not set
# CONFIG_SCSI_CXGB3_ISCSI is not set
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_AIC94XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_ARCMSR is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
# CONFIG_SCSI_HPTIOP is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_LIBFC is not set
# CONFIG_FCOE is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not set
# CONFIG_SCSI_MVSAS is not set
# CONFIG_SCSI_STEX is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_QLA_ISCSI is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set
# CONFIG_SCSI_SRP is not set
# CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
# CONFIG_SCSI_DH is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_ACPI=y
CONFIG_SATA_PMP=y
CONFIG_SATA_AHCI=y
# CONFIG_SATA_SIL24 is not set
CONFIG_ATA_SFF=y
# CONFIG_SATA_SVW is not set
CONFIG_ATA_PIIX=y
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SX4 is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_PATA_ACPI is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
# CONFIG_PATA_ATIIXP is not set
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CS5535 is not set
# CONFIG_PATA_CS5536 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
CONFIG_ATA_GENERIC=y
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_MARVELL is not set
CONFIG_PATA_MPIIX=y
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PCMCIA is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set
# CONFIG_PATA_SCH is not set
# CONFIG_MD is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#

#
# Enable only one of the two stacks, unless you know what you are doing
#
# CONFIG_FIREWIRE is not set
CONFIG_IEEE1394=y
CONFIG_IEEE1394_OHCI1394=y
# CONFIG_IEEE1394_PCILYNX is not set
CONFIG_IEEE1394_SBP2=y
CONFIG_IEEE1394_SBP2_PHYS_DMA=y
# CONFIG_IEEE1394_ETH1394_ROM_ENTRY is not set
# CONFIG_IEEE1394_ETH1394 is not set
# CONFIG_IEEE1394_RAWIO is not set
# CONFIG_IEEE1394_VIDEO1394 is not set
# CONFIG_IEEE1394_DV1394 is not set
# CONFIG_IEEE1394_VERBOSEDEBUG is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
# CONFIG_BONDING is not set
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=y
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
# CONFIG_PHYLIB is not set
CONFIG_NET_ETHERNET=y
CONFIG_MII=y
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=y
# CONFIG_TYPHOON is not set
# CONFIG_DNET is not set
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
# CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
# CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
CONFIG_8139CP=y
CONFIG_8139TOO=y
# CONFIG_8139TOO_PIO is not set
CONFIG_8139TOO_TUNE_TWISTER=y
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_R6040 is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SMSC9420 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_SC92031 is not set
# CONFIG_NET_POCKET is not set
# CONFIG_ATL2 is not set
CONFIG_NETDEV_1000=y
CONFIG_ACENIC=y
# CONFIG_ACENIC_OMIT_TIGON_I is not set
# CONFIG_DL2K is not set
CONFIG_E1000=y
# CONFIG_E1000E is not set
# CONFIG_IP1000 is not set
# CONFIG_IGB is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_QLA3XXX is not set
# CONFIG_ATL1 is not set
# CONFIG_ATL1E is not set
# CONFIG_ATL1C is not set
# CONFIG_JME is not set
CONFIG_NETDEV_10000=y
# CONFIG_CHELSIO_T1 is not set
CONFIG_CHELSIO_T3_DEPENDS=y
# CONFIG_CHELSIO_T3 is not set
# CONFIG_ENIC is not set
# CONFIG_IXGBE is not set
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set
# CONFIG_MYRI10GE is not set
# CONFIG_NETXEN_NIC is not set
# CONFIG_NIU is not set
# CONFIG_MLX4_EN is not set
# CONFIG_MLX4_CORE is not set
# CONFIG_TEHUTI is not set
# CONFIG_BNX2X is not set
# CONFIG_QLGE is not set
# CONFIG_SFC is not set
# CONFIG_BE2NET is not set
# CONFIG_TR is not set

#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set
# CONFIG_IWLWIFI_LEDS is not set

#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
CONFIG_NET_PCMCIA=y
# CONFIG_PCMCIA_3C589 is not set
# CONFIG_PCMCIA_3C574 is not set
# CONFIG_PCMCIA_FMVJ18X is not set
# CONFIG_PCMCIA_PCNET is not set
# CONFIG_PCMCIA_NMCLAN is not set
# CONFIG_PCMCIA_SMC91C92 is not set
# CONFIG_PCMCIA_XIRC2PS is not set
# CONFIG_PCMCIA_AXNET is not set
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=y
# CONFIG_PPP_MULTILINK is not set
# CONFIG_PPP_FILTER is not set
# CONFIG_PPP_ASYNC is not set
# CONFIG_PPP_SYNC_TTY is not set
# CONFIG_PPP_DEFLATE is not set
# CONFIG_PPP_BSDCOMP is not set
# CONFIG_PPP_MPPE is not set
# CONFIG_PPPOE is not set
# CONFIG_PPPOL2TP is not set
# CONFIG_SLIP is not set
CONFIG_SLHC=y
# CONFIG_NET_FC is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
CONFIG_VIRTIO_NET=y
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_FF_MEMLESS=m
# CONFIG_INPUT_POLLDEV is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
# CONFIG_MOUSE_PS2_ELANTECH is not set
# CONFIG_MOUSE_PS2_TOUCHKIT is not set
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_BCM5974 is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=y
# CONFIG_INPUT_WISTRON_BTNS is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
# CONFIG_INPUT_CM109 is not set
# CONFIG_INPUT_UINPUT is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=y
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
CONFIG_DEVKMEM=y
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NOZOMI is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
# CONFIG_SERIAL_8250_CS is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
# CONFIG_SERIAL_8250_SHARE_IRQ is not set
CONFIG_SERIAL_8250_DETECT_IRQ=y
# CONFIG_SERIAL_8250_RSA is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_CONSOLE_POLL=y
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_PRINTER is not set
# CONFIG_PPDEV is not set
# CONFIG_VIRTIO_CONSOLE is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_HW_RANDOM is not set
CONFIG_NVRAM=y
CONFIG_RTC=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not set
# CONFIG_CARDMAN_4000 is not set
# CONFIG_CARDMAN_4040 is not set
# CONFIG_IPWIRELESS is not set
# CONFIG_MWAVE is not set
# CONFIG_PC8736x_GPIO is not set
# CONFIG_NSC_GPIO is not set
# CONFIG_CS5535_GPIO is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
# CONFIG_I2C_CHARDEV is not set
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_ALGOBIT=y

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
CONFIG_I2C_I801=y
# CONFIG_I2C_ISCH is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_SIMTEC is not set

#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_PARPORT is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set

#
# Graphics adapter I2C/DDC channel drivers
#
# CONFIG_I2C_VOODOO3 is not set

#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_STUB is not set
# CONFIG_SCx200_ACB is not set

#
# Miscellaneous I2C Chip support
#
# CONFIG_DS1682 is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_PCF8575 is not set
# CONFIG_SENSORS_PCA9539 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
# CONFIG_SPI is not set
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
# CONFIG_PDA_POWER is not set
# CONFIG_BATTERY_DS2760 is not set
# CONFIG_BATTERY_BQ27x00 is not set
CONFIG_HWMON=y
# CONFIG_HWMON_VID is not set
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_ABITUGURU3 is not set
# CONFIG_SENSORS_AD7414 is not set
# CONFIG_SENSORS_AD7418 is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ADT7462 is not set
# CONFIG_SENSORS_ADT7470 is not set
# CONFIG_SENSORS_ADT7473 is not set
# CONFIG_SENSORS_ADT7475 is not set
# CONFIG_SENSORS_K8TEMP is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_F71882FG is not set
# CONFIG_SENSORS_F75375S is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_FSCPOS is not set
# CONFIG_SENSORS_FSCHMD is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_CORETEMP is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_LM93 is not set
# CONFIG_SENSORS_LTC4245 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_PC87427 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_DME1737 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47M192 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_ADS7828 is not set
# CONFIG_SENSORS_THMC50 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
# CONFIG_SENSORS_VT8231 is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83791D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83L786NG is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_SENSORS_LIS3LV02D is not set
# CONFIG_SENSORS_APPLESMC is not set
# CONFIG_HWMON_DEBUG_CHIP is not set
CONFIG_THERMAL=y
# CONFIG_THERMAL_HWMON is not set
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y

#
# Sonics Silicon Backplane
#
# CONFIG_SSB is not set

#
# Multifunction device drivers
#
# CONFIG_MFD_CORE is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_TWL4030_CORE is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_PMIC_DA903X is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM8350_I2C is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_REGULATOR is not set

#
# Multimedia devices
#

#
# Multimedia core support
#
# CONFIG_VIDEO_DEV is not set
# CONFIG_DVB_CORE is not set
# CONFIG_VIDEO_MEDIA is not set

#
# Multimedia drivers
#
CONFIG_DAB=y
# CONFIG_USB_DABUSB is not set

#
# Graphics support
#
CONFIG_AGP=y
# CONFIG_AGP_ALI is not set
# CONFIG_AGP_ATI is not set
# CONFIG_AGP_AMD is not set
CONFIG_AGP_AMD64=y
CONFIG_AGP_INTEL=y
# CONFIG_AGP_NVIDIA is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_AGP_EFFICEON is not set
CONFIG_DRM=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=y
CONFIG_DRM_I810=y
CONFIG_DRM_I830=y
# CONFIG_DRM_I915 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
# CONFIG_FB is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set

#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_VGACON_SOFT_SCROLLBACK is not set
CONFIG_DUMMY_CONSOLE=y
CONFIG_SOUND=y
CONFIG_SOUND_OSS_CORE=y
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_RAWMIDI=y
CONFIG_SND_SEQUENCER=y
CONFIG_SND_SEQ_DUMMY=y
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=y
CONFIG_SND_PCM_OSS=y
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=y
CONFIG_SND_SEQ_RTCTIMER_DEFAULT=y
# CONFIG_SND_DYNAMIC_MINORS is not set
CONFIG_SND_SUPPORT_OLD_API=y
CONFIG_SND_VERBOSE_PROCFS=y
CONFIG_SND_VERBOSE_PRINTK=y
# CONFIG_SND_DEBUG is not set
CONFIG_SND_VMASTER=y
CONFIG_SND_AC97_CODEC=y
CONFIG_SND_DRIVERS=y
# CONFIG_SND_DUMMY is not set
CONFIG_SND_VIRMIDI=y
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_MTS64 is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set
# CONFIG_SND_PORTMAN2X4 is not set
# CONFIG_SND_AC97_POWER_SAVE is not set
CONFIG_SND_PCI=y
# CONFIG_SND_AD1889 is not set
# CONFIG_SND_ALS300 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AW2 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_OXYGEN is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS5530 is not set
# CONFIG_SND_CS5535AUDIO is not set
# CONFIG_SND_DARLA20 is not set
# CONFIG_SND_GINA20 is not set
# CONFIG_SND_LAYLA20 is not set
# CONFIG_SND_DARLA24 is not set
# CONFIG_SND_GINA24 is not set
# CONFIG_SND_LAYLA24 is not set
# CONFIG_SND_MONA is not set
# CONFIG_SND_MIA is not set
# CONFIG_SND_ECHO3G is not set
# CONFIG_SND_INDIGO is not set
# CONFIG_SND_INDIGOIO is not set
# CONFIG_SND_INDIGODJ is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_HDA_INTEL is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
# CONFIG_SND_HIFIER is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
CONFIG_SND_INTEL8X0=y
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_PCXHR is not set
# CONFIG_SND_RIPTIDE is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_SIS7019 is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VIRTUOSO is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_YMFPCI is not set
CONFIG_SND_USB=y
# CONFIG_SND_USB_AUDIO is not set
# CONFIG_SND_USB_USX2Y is not set
# CONFIG_SND_USB_CAIAQ is not set
# CONFIG_SND_USB_US122L is not set
CONFIG_SND_PCMCIA=y
# CONFIG_SND_VXPOCKET is not set
# CONFIG_SND_PDAUDIOCF is not set
# CONFIG_SND_SOC is not set
# CONFIG_SOUND_PRIME is not set
CONFIG_AC97_BUS=y
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
# CONFIG_HID_DEBUG is not set
# CONFIG_HIDRAW is not set

#
# USB Input Devices
#
CONFIG_USB_HID=y
# CONFIG_HID_PID is not set
# CONFIG_USB_HIDDEV is not set

#
# Special HID drivers
#
CONFIG_HID_COMPAT=y
CONFIG_HID_A4TECH=y
CONFIG_HID_APPLE=y
CONFIG_HID_BELKIN=y
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
CONFIG_HID_CYPRESS=y
CONFIG_HID_EZKEY=y
CONFIG_HID_GYRATION=y
CONFIG_HID_LOGITECH=y
# CONFIG_LOGITECH_FF is not set
# CONFIG_LOGIRUMBLEPAD2_FF is not set
CONFIG_HID_MICROSOFT=y
CONFIG_HID_MONTEREY=y
CONFIG_HID_NTRIG=y
CONFIG_HID_PANTHERLORD=y
# CONFIG_PANTHERLORD_FF is not set
CONFIG_HID_PETALYNX=y
CONFIG_HID_SAMSUNG=y
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
# CONFIG_GREENASIA_FF is not set
CONFIG_HID_TOPSEED=y
CONFIG_THRUSTMASTER_FF=m
CONFIG_ZEROPLUS_FF=m
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
# CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DEVICE_CLASS=y
CONFIG_USB_DYNAMIC_MINORS=y
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set
CONFIG_USB_MON=y
# CONFIG_USB_WUSB is not set
# CONFIG_USB_WUSB_CBAF is not set

#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_ISP1760_HCD is not set
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=y
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_WHCI_HCD is not set
# CONFIG_USB_HWA_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
CONFIG_USB_PRINTER=y
# CONFIG_USB_WDM is not set
# CONFIG_USB_TMC is not set

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may also be needed;
#

#
# see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_STORAGE_ONETOUCH is not set
# CONFIG_USB_STORAGE_KARMA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_LIBUSUAL is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set

#
# USB port drivers
#
# CONFIG_USB_USS720 is not set
# CONFIG_USB_SERIAL is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_BERRY_CHARGE is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGET is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_VST is not set
# CONFIG_USB_GADGET is not set

#
# OTG and related infrastructure
#
# CONFIG_UWB is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_ACCESSIBILITY is not set
CONFIG_INFINIBAND=y
# CONFIG_INFINIBAND_USER_MAD is not set
# CONFIG_INFINIBAND_USER_ACCESS is not set
CONFIG_INFINIBAND_ADDR_TRANS=y
# CONFIG_INFINIBAND_MTHCA is not set
# CONFIG_INFINIBAND_AMSO1100 is not set
# CONFIG_MLX4_INFINIBAND is not set
# CONFIG_INFINIBAND_NES is not set
# CONFIG_INFINIBAND_IPOIB is not set
# CONFIG_INFINIBAND_SRP is not set
# CONFIG_INFINIBAND_ISER is not set
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_MM_EDAC=y
# CONFIG_EDAC_AMD76X is not set
# CONFIG_EDAC_E7XXX is not set
# CONFIG_EDAC_E752X is not set
# CONFIG_EDAC_I82875P is not set
# CONFIG_EDAC_I82975X is not set
# CONFIG_EDAC_I3000 is not set
# CONFIG_EDAC_X38 is not set
# CONFIG_EDAC_I5400 is not set
# CONFIG_EDAC_I82860 is not set
# CONFIG_EDAC_R82600 is not set
# CONFIG_EDAC_I5000 is not set
# CONFIG_EDAC_I5100 is not set
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set
# CONFIG_AUXDISPLAY is not set
# CONFIG_UIO is not set
# CONFIG_STAGING is not set
CONFIG_X86_PLATFORM_DEVICES=y
# CONFIG_ASUS_LAPTOP is not set
# CONFIG_TC1100_WMI is not set
# CONFIG_SONY_LAPTOP is not set
# CONFIG_THINKPAD_ACPI is not set
# CONFIG_INTEL_MENLOW is not set
# CONFIG_EEEPC_LAPTOP is not set
# CONFIG_ACPI_WMI is not set
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_TOSHIBA is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
CONFIG_DMIID=y
# CONFIG_ISCSI_IBFT_FIND is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_EXT4_FS=y
# CONFIG_EXT4DEV_COMPAT is not set
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
CONFIG_JBD=y
CONFIG_JBD_DEBUG=y
CONFIG_JBD2=y
CONFIG_JBD2_DEBUG=y
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
# CONFIG_REISERFS_FS_SECURITY is not set
CONFIG_JFS_FS=y
CONFIG_JFS_POSIX_ACL=y
# CONFIG_JFS_SECURITY is not set
# CONFIG_JFS_DEBUG is not set
# CONFIG_JFS_STATISTICS is not set
CONFIG_FS_POSIX_ACL=y
CONFIG_FILE_LOCKING=y
CONFIG_XFS_FS=y
# CONFIG_XFS_QUOTA is not set
CONFIG_XFS_POSIX_ACL=y
# CONFIG_XFS_RT is not set
# CONFIG_XFS_DEBUG is not set
# CONFIG_GFS2_FS is not set
CONFIG_OCFS2_FS=y
CONFIG_OCFS2_FS_O2CB=y
CONFIG_OCFS2_FS_USERSPACE_CLUSTER=y
CONFIG_OCFS2_FS_STATS=y
CONFIG_OCFS2_DEBUG_MASKLOG=y
# CONFIG_OCFS2_DEBUG_FS is not set
# CONFIG_OCFS2_FS_POSIX_ACL is not set
# CONFIG_BTRFS_FS is not set
CONFIG_DNOTIFY=y
CONFIG_INOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_QUOTA=y
# CONFIG_QUOTA_NETLINK_INTERFACE is not set
CONFIG_PRINT_QUOTA_WARNING=y
CONFIG_QUOTA_TREE=y
# CONFIG_QFMT_V1 is not set
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_CONFIGFS_FS=y
CONFIG_MISC_FILESYSTEMS=y
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_ECRYPT_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_SQUASHFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_OMFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFSD=y
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
# CONFIG_NFSD_V4_1 is not set
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_NFS_ACL_SUPPORT=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_SUNRPC_XPRT_RDMA=y
# CONFIG_SUNRPC_REGISTER_V4 is not set
CONFIG_RPCSEC_GSS_KRB5=y
CONFIG_RPCSEC_GSS_SPKM3=y
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
# CONFIG_EFI_PARTITION is not set
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_UTF8 is not set
CONFIG_DLM=y
CONFIG_DLM_DEBUG=y

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=1024
CONFIG_MAGIC_SYSRQ=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
# CONFIG_TIMER_STATS is not set
CONFIG_DEBUG_OBJECTS=y
# CONFIG_DEBUG_OBJECTS_SELFTEST is not set
CONFIG_DEBUG_OBJECTS_FREE=y
CONFIG_DEBUG_OBJECTS_TIMERS=y
CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT=1
CONFIG_DEBUG_SLAB=y
CONFIG_DEBUG_SLAB_LEAK=y
CONFIG_DEBUG_PREEMPT=y
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCKDEP=y
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_TRACE_IRQFLAGS=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_HIGHMEM is not set
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_VM=y
# CONFIG_DEBUG_VIRTUAL is not set
CONFIG_DEBUG_WRITECOUNT=y
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_DEBUG_LIST=y
CONFIG_DEBUG_SG=y
# CONFIG_DEBUG_NOTIFIERS is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_CPU_STALL_DETECTOR is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_LKDTM is not set
CONFIG_FAULT_INJECTION=y
CONFIG_FAILSLAB=y
CONFIG_FAIL_PAGE_ALLOC=y
CONFIG_FAIL_MAKE_REQUEST=y
# CONFIG_FAIL_IO_TIMEOUT is not set
CONFIG_FAULT_INJECTION_DEBUG_FS=y
# CONFIG_FAULT_INJECTION_STACKTRACE_FILTER is not set
CONFIG_LATENCYTOP=y
CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_HW_BRANCH_TRACER=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_RING_BUFFER=y
CONFIG_TRACING=y

#
# Tracers
#
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
# CONFIG_IRQSOFF_TRACER is not set
# CONFIG_PREEMPT_TRACER is not set
CONFIG_SYSPROF_TRACER=y
CONFIG_SCHED_TRACER=y
CONFIG_CONTEXT_SWITCH_TRACER=y
# CONFIG_BOOT_TRACER is not set
# CONFIG_TRACE_BRANCH_PROFILING is not set
# CONFIG_POWER_TRACER is not set
CONFIG_STACK_TRACER=y
# CONFIG_HW_BRANCH_TRACER is not set
CONFIG_DYNAMIC_FTRACE=y
CONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_MMIOTRACE is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
CONFIG_DYNAMIC_PRINTK_DEBUG=y
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_KGDB=y
CONFIG_KGDB_SERIAL_CONSOLE=y
# CONFIG_KGDB_TESTS is not set
# CONFIG_STRICT_DEVMEM is not set
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_EARLY_PRINTK=y
# CONFIG_EARLY_PRINTK_DBGP is not set
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_STACK_USAGE=y
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_X86_PTDUMP is not set
CONFIG_DEBUG_RODATA=y
# CONFIG_DEBUG_RODATA_TEST is not set
# CONFIG_DEBUG_NX_TEST is not set
# CONFIG_4KSTACKS is not set
CONFIG_DOUBLEFAULT=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
# CONFIG_IO_DELAY_0X80 is not set
CONFIG_IO_DELAY_0XED=y
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=1
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set

#
# Security options
#
CONFIG_KEYS=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
CONFIG_SECURITY=y
# CONFIG_SECURITYFS is not set
# CONFIG_SECURITY_NETWORK is not set
# CONFIG_SECURITY_PATH is not set
# CONFIG_SECURITY_FILE_CAPABILITIES is not set
# CONFIG_SECURITY_ROOTPLUG is not set
CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR=0
CONFIG_CRYPTO=y

#
# Crypto core or helper
#
# CONFIG_CRYPTO_FIPS is not set
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_GF128MUL is not set
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_CRYPTD is not set
# CONFIG_CRYPTO_AUTHENC is not set
# CONFIG_CRYPTO_TEST is not set

#
# Authenticated Encryption with Associated Data
#
# CONFIG_CRYPTO_CCM is not set
# CONFIG_CRYPTO_GCM is not set
# CONFIG_CRYPTO_SEQIV is not set

#
# Block modes
#
CONFIG_CRYPTO_CBC=y
# CONFIG_CRYPTO_CTR is not set
# CONFIG_CRYPTO_CTS is not set
CONFIG_CRYPTO_ECB=y
# CONFIG_CRYPTO_LRW is not set
# CONFIG_CRYPTO_PCBC is not set
# CONFIG_CRYPTO_XTS is not set

#
# Hash modes
#
CONFIG_CRYPTO_HMAC=y
# CONFIG_CRYPTO_XCBC is not set

#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
# CONFIG_CRYPTO_CRC32C_INTEL is not set
# CONFIG_CRYPTO_MD4 is not set
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=y
# CONFIG_CRYPTO_RMD128 is not set
# CONFIG_CRYPTO_RMD160 is not set
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not set
CONFIG_CRYPTO_SHA1=y
# CONFIG_CRYPTO_SHA256 is not set
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_TGR192 is not set
# CONFIG_CRYPTO_WP512 is not set

#
# Ciphers
#
CONFIG_CRYPTO_AES=y
# CONFIG_CRYPTO_AES_586 is not set
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_ARC4=y
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_CAMELLIA is not set
CONFIG_CRYPTO_CAST5=y
# CONFIG_CRYPTO_CAST6 is not set
CONFIG_CRYPTO_DES=y
# CONFIG_CRYPTO_FCRYPT is not set
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_SALSA20 is not set
# CONFIG_CRYPTO_SALSA20_586 is not set
# CONFIG_CRYPTO_SEED is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_TWOFISH_586 is not set

#
# Compression
#
# CONFIG_CRYPTO_DEFLATE is not set
# CONFIG_CRYPTO_LZO is not set

#
# Random Number Generation
#
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_HW=y
# CONFIG_CRYPTO_DEV_PADLOCK is not set
# CONFIG_CRYPTO_DEV_GEODE is not set
# CONFIG_CRYPTO_DEV_HIFN_795X is not set
CONFIG_HAVE_KVM=y
CONFIG_VIRTUALIZATION=y
# CONFIG_KVM is not set
# CONFIG_LGUEST is not set
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
# CONFIG_VIRTIO_BALLOON is not set

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
# CONFIG_CRC_CCITT is not set
CONFIG_CRC16=y
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=y
CONFIG_CRC32=y
# CONFIG_CRC7 is not set
CONFIG_LIBCRC32C=y
CONFIG_ZLIB_INFLATE=y
CONFIG_PLIST=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Benny Halevy]

On Mar. 30, 2009, 22:07 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Mon, Mar 30, 2009 at 10:03:39PM +0300, Benny Halevy wrote:
>> On Mar. 30, 2009, 21:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
>>>> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
>>>>>> Hi Bruce,
>>>>>>
>>>>>> Here's the latest server patches implementing the NFSv4.1
>>>>>> Sessions features.
>>>>>>
>>>>>> This patchset is based over your for-2.6.30 branch
>>>>>> and is also available from
>>>>>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
>>>>> There's a few merge conflicts with my current for-2.6.30--probably my
>>>>> fault for not pushing that out recently enough, apologies.  Would you
>>>>> mind updating?
>>>>>
>>>>> --b.
>>>>>
>>>> Sure. Here's a rebased version with two minor changes (see below)
>>> On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
>>> dereference in the callback code.  Looks like it probably happened while
>>> running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so
>> Weird. nfs4_xdr_dec_cb_recall+0x4e doesn't seem like a valid IP
>> for nfsd.ko @a564667.  Bruce, can you please send me you .config file?
> 
> I'm building without CONFIG_NFSD_V4_1 for now; .config appended.

It looks like rpc_res would be NULL without CONFIG_NFSD_V4_1.
Does the following patch help?

Benny

git diff --stat -p
 fs/nfsd/nfs4callback.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 02e0a61..10c6a80 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -441,9 +441,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
-	status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
-	if (status)
-		goto out;
+	if (rpc_res && rpc_res->res_seq) {
+		status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
+		if (status)
+			goto out;
+	}
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;


> 
> Start of my nfs4_xdr_dec_cb_recall, for what it's worth:
> 
> c03e4dd0 <nfs4_xdr_dec_cb_recall>:
> c03e4dd0:       55                      push   %ebp
> c03e4dd1:       89 e5                   mov    %esp,%ebp
> c03e4dd3:       83 ec 38                sub    $0x38,%esp
> c03e4dd6:       89 5d f4                mov    %ebx,-0xc(%ebp)
> c03e4dd9:       89 75 f8                mov    %esi,-0x8(%ebp)
> c03e4ddc:       89 7d fc                mov    %edi,-0x4(%ebp)
> c03e4ddf:       e8 8c ed e1 ff          call   c0203b70 <mcount>
> c03e4de4:       89 d3                   mov    %edx,%ebx
> c03e4de6:       89 ce                   mov    %ecx,%esi
> c03e4de8:       8d 50 2c                lea    0x2c(%eax),%edx
> c03e4deb:       89 d9                   mov    %ebx,%ecx
> c03e4ded:       8d 45 e4                lea    -0x1c(%ebp),%eax
> c03e4df0:       e8 eb c0 43 00          call   c0820ee0 <xdr_init_decode>
> c03e4df5:       ba 08 00 00 00          mov    $0x8,%edx
> c03e4dfa:       8d 45 e4                lea    -0x1c(%ebp),%eax
> c03e4dfd:       e8 0e c1 43 00          call   c0820f10 <xdr_inline_decode>
> c03e4e02:       85 c0                   test   %eax,%eax
> c03e4e04:       74 3a                   je     c03e4e40 <nfs4_xdr_dec_cb_recall+0x70>
> c03e4e06:       8b 50 04                mov    0x4(%eax),%edx
> c03e4e09:       8d 45 e4                lea    -0x1c(%ebp),%eax
> c03e4e0c:       0f ca                   bswap  %edx
> c03e4e0e:       83 c2 04                add    $0x4,%edx
> c03e4e11:       e8 fa c0 43 00          call   c0820f10 <xdr_inline_decode>
> c03e4e16:       85 c0                   test   %eax,%eax
> c03e4e18:       0f 84 8a 00 00 00       je     c03e4ea8 <nfs4_xdr_dec_cb_recall+0xd8>
> --> c03e4e1e:   8b 06                   mov    (%esi),%eax
> c03e4e20:       8b 00                   mov    (%eax),%eax
> c03e4e22:       85 c0                   test   %eax,%eax
> c03e4e24:       75 32                   jne    c03e4e58 <nfs4_xdr_dec_cb_recall+0x88>
> c03e4e26:       ba 04 00 00 00          mov    $0x4,%edx
> c03e4e2b:       8d 45 e4                lea    -0x1c(%ebp),%eax
> c03e4e2e:       e8 cd fe ff ff          call   c03e4d00 <decode_cb_op_hdr>
> c03e4e33:       8b 5d f4                mov    -0xc(%ebp),%ebx
> c03e4e36:       8b 75 f8                mov    -0x8(%ebp),%esi
> c03e4e39:       8b 7d fc                mov    -0x4(%ebp),%edi
> c03e4e3c:       89 ec                   mov    %ebp,%esp
> c03e4e3e:       5d                      pop    %ebp
> c03e4e3f:       c3                      ret
> ...
> 
> 
> --b.
> 
> #
> # Automatically generated make config: don't edit
> # Linux kernel version: 2.6.29-rc8
> # Mon Mar 30 14:07:10 2009
> #
> # CONFIG_64BIT is not set
> CONFIG_X86_32=y
> # CONFIG_X86_64 is not set
> CONFIG_X86=y
> CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
> CONFIG_GENERIC_TIME=y
> CONFIG_GENERIC_CMOS_UPDATE=y
> CONFIG_CLOCKSOURCE_WATCHDOG=y
> CONFIG_GENERIC_CLOCKEVENTS=y
> CONFIG_LOCKDEP_SUPPORT=y
> CONFIG_STACKTRACE_SUPPORT=y
> CONFIG_HAVE_LATENCYTOP_SUPPORT=y
> CONFIG_FAST_CMPXCHG_LOCAL=y
> CONFIG_MMU=y
> CONFIG_ZONE_DMA=y
> CONFIG_GENERIC_ISA_DMA=y
> CONFIG_GENERIC_IOMAP=y
> CONFIG_GENERIC_BUG=y
> CONFIG_GENERIC_HWEIGHT=y
> CONFIG_ARCH_MAY_HAVE_PC_FDC=y
> # CONFIG_RWSEM_GENERIC_SPINLOCK is not set
> CONFIG_RWSEM_XCHGADD_ALGORITHM=y
> CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
> CONFIG_GENERIC_CALIBRATE_DELAY=y
> # CONFIG_GENERIC_TIME_VSYSCALL is not set
> CONFIG_ARCH_HAS_CPU_RELAX=y
> CONFIG_ARCH_HAS_DEFAULT_IDLE=y
> CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
> # CONFIG_HAVE_SETUP_PER_CPU_AREA is not set
> # CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set
> CONFIG_ARCH_HIBERNATION_POSSIBLE=y
> CONFIG_ARCH_SUSPEND_POSSIBLE=y
> # CONFIG_ZONE_DMA32 is not set
> CONFIG_ARCH_POPULATES_NODE_MAP=y
> # CONFIG_AUDIT_ARCH is not set
> CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
> CONFIG_GENERIC_HARDIRQS=y
> CONFIG_GENERIC_IRQ_PROBE=y
> CONFIG_X86_BIOS_REBOOT=y
> CONFIG_KTIME_SCALAR=y
> CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
> 
> #
> # General setup
> #
> CONFIG_EXPERIMENTAL=y
> CONFIG_BROKEN_ON_SMP=y
> CONFIG_LOCK_KERNEL=y
> CONFIG_INIT_ENV_ARG_LIMIT=32
> CONFIG_LOCALVERSION=""
> CONFIG_LOCALVERSION_AUTO=y
> CONFIG_SWAP=y
> CONFIG_SYSVIPC=y
> CONFIG_SYSVIPC_SYSCTL=y
> CONFIG_POSIX_MQUEUE=y
> CONFIG_BSD_PROCESS_ACCT=y
> CONFIG_BSD_PROCESS_ACCT_V3=y
> # CONFIG_TASKSTATS is not set
> # CONFIG_AUDIT is not set
> 
> #
> # RCU Subsystem
> #
> CONFIG_CLASSIC_RCU=y
> # CONFIG_TREE_RCU is not set
> # CONFIG_PREEMPT_RCU is not set
> # CONFIG_TREE_RCU_TRACE is not set
> # CONFIG_PREEMPT_RCU_TRACE is not set
> CONFIG_IKCONFIG=y
> CONFIG_IKCONFIG_PROC=y
> CONFIG_LOG_BUF_SHIFT=20
> CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
> CONFIG_GROUP_SCHED=y
> CONFIG_FAIR_GROUP_SCHED=y
> # CONFIG_RT_GROUP_SCHED is not set
> CONFIG_USER_SCHED=y
> # CONFIG_CGROUP_SCHED is not set
> # CONFIG_CGROUPS is not set
> CONFIG_SYSFS_DEPRECATED=y
> CONFIG_SYSFS_DEPRECATED_V2=y
> CONFIG_RELAY=y
> CONFIG_NAMESPACES=y
> # CONFIG_UTS_NS is not set
> # CONFIG_IPC_NS is not set
> # CONFIG_USER_NS is not set
> # CONFIG_PID_NS is not set
> # CONFIG_NET_NS is not set
> # CONFIG_BLK_DEV_INITRD is not set
> # CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
> CONFIG_SYSCTL=y
> CONFIG_ANON_INODES=y
> # CONFIG_EMBEDDED is not set
> CONFIG_UID16=y
> CONFIG_SYSCTL_SYSCALL=y
> CONFIG_KALLSYMS=y
> CONFIG_KALLSYMS_ALL=y
> # CONFIG_KALLSYMS_EXTRA_PASS is not set
> CONFIG_HOTPLUG=y
> CONFIG_PRINTK=y
> CONFIG_BUG=y
> CONFIG_ELF_CORE=y
> CONFIG_PCSPKR_PLATFORM=y
> CONFIG_BASE_FULL=y
> CONFIG_FUTEX=y
> CONFIG_EPOLL=y
> CONFIG_SIGNALFD=y
> CONFIG_TIMERFD=y
> CONFIG_EVENTFD=y
> CONFIG_SHMEM=y
> CONFIG_AIO=y
> CONFIG_VM_EVENT_COUNTERS=y
> CONFIG_PCI_QUIRKS=y
> CONFIG_COMPAT_BRK=y
> CONFIG_SLAB=y
> # CONFIG_SLUB is not set
> # CONFIG_SLOB is not set
> CONFIG_PROFILING=y
> CONFIG_TRACEPOINTS=y
> CONFIG_MARKERS=y
> CONFIG_OPROFILE=y
> CONFIG_HAVE_OPROFILE=y
> CONFIG_KPROBES=y
> CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
> CONFIG_KRETPROBES=y
> CONFIG_HAVE_IOREMAP_PROT=y
> CONFIG_HAVE_KPROBES=y
> CONFIG_HAVE_KRETPROBES=y
> CONFIG_HAVE_ARCH_TRACEHOOK=y
> CONFIG_HAVE_GENERIC_DMA_COHERENT=y
> CONFIG_SLABINFO=y
> CONFIG_RT_MUTEXES=y
> CONFIG_BASE_SMALL=0
> CONFIG_MODULES=y
> # CONFIG_MODULE_FORCE_LOAD is not set
> CONFIG_MODULE_UNLOAD=y
> # CONFIG_MODULE_FORCE_UNLOAD is not set
> # CONFIG_MODVERSIONS is not set
> # CONFIG_MODULE_SRCVERSION_ALL is not set
> CONFIG_BLOCK=y
> CONFIG_LBD=y
> # CONFIG_BLK_DEV_IO_TRACE is not set
> # CONFIG_BLK_DEV_BSG is not set
> # CONFIG_BLK_DEV_INTEGRITY is not set
> 
> #
> # IO Schedulers
> #
> CONFIG_IOSCHED_NOOP=y
> CONFIG_IOSCHED_AS=y
> CONFIG_IOSCHED_DEADLINE=y
> CONFIG_IOSCHED_CFQ=y
> CONFIG_DEFAULT_AS=y
> # CONFIG_DEFAULT_DEADLINE is not set
> # CONFIG_DEFAULT_CFQ is not set
> # CONFIG_DEFAULT_NOOP is not set
> CONFIG_DEFAULT_IOSCHED="anticipatory"
> CONFIG_FREEZER=y
> 
> #
> # Processor type and features
> #
> # CONFIG_NO_HZ is not set
> # CONFIG_HIGH_RES_TIMERS is not set
> CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
> # CONFIG_SMP is not set
> CONFIG_X86_PC=y
> # CONFIG_X86_ELAN is not set
> # CONFIG_X86_VOYAGER is not set
> # CONFIG_X86_GENERICARCH is not set
> # CONFIG_X86_VSMP is not set
> # CONFIG_X86_RDC321X is not set
> CONFIG_SCHED_OMIT_FRAME_POINTER=y
> # CONFIG_PARAVIRT_GUEST is not set
> # CONFIG_MEMTEST is not set
> # CONFIG_M386 is not set
> # CONFIG_M486 is not set
> # CONFIG_M586 is not set
> # CONFIG_M586TSC is not set
> # CONFIG_M586MMX is not set
> CONFIG_M686=y
> # CONFIG_MPENTIUMII is not set
> # CONFIG_MPENTIUMIII is not set
> # CONFIG_MPENTIUMM is not set
> # CONFIG_MPENTIUM4 is not set
> # CONFIG_MK6 is not set
> # CONFIG_MK7 is not set
> # CONFIG_MK8 is not set
> # CONFIG_MCRUSOE is not set
> # CONFIG_MEFFICEON is not set
> # CONFIG_MWINCHIPC6 is not set
> # CONFIG_MWINCHIP3D is not set
> # CONFIG_MGEODEGX1 is not set
> # CONFIG_MGEODE_LX is not set
> # CONFIG_MCYRIXIII is not set
> # CONFIG_MVIAC3_2 is not set
> # CONFIG_MVIAC7 is not set
> # CONFIG_MPSC is not set
> # CONFIG_MCORE2 is not set
> # CONFIG_GENERIC_CPU is not set
> # CONFIG_X86_GENERIC is not set
> CONFIG_X86_CPU=y
> CONFIG_X86_CMPXCHG=y
> CONFIG_X86_L1_CACHE_SHIFT=5
> CONFIG_X86_XADD=y
> # CONFIG_X86_PPRO_FENCE is not set
> CONFIG_X86_WP_WORKS_OK=y
> CONFIG_X86_INVLPG=y
> CONFIG_X86_BSWAP=y
> CONFIG_X86_POPAD_OK=y
> CONFIG_X86_USE_PPRO_CHECKSUM=y
> CONFIG_X86_TSC=y
> CONFIG_X86_CMOV=y
> CONFIG_X86_MINIMUM_CPU_FAMILY=4
> CONFIG_X86_DEBUGCTLMSR=y
> CONFIG_CPU_SUP_INTEL=y
> CONFIG_CPU_SUP_CYRIX_32=y
> CONFIG_CPU_SUP_AMD=y
> CONFIG_CPU_SUP_CENTAUR_32=y
> CONFIG_CPU_SUP_TRANSMETA_32=y
> CONFIG_CPU_SUP_UMC_32=y
> CONFIG_X86_DS=y
> CONFIG_X86_PTRACE_BTS=y
> CONFIG_HPET_TIMER=y
> CONFIG_HPET_EMULATE_RTC=y
> CONFIG_DMI=y
> # CONFIG_IOMMU_HELPER is not set
> # CONFIG_IOMMU_API is not set
> CONFIG_NR_CPUS=1
> # CONFIG_PREEMPT_NONE is not set
> # CONFIG_PREEMPT_VOLUNTARY is not set
> CONFIG_PREEMPT=y
> # CONFIG_X86_UP_APIC is not set
> CONFIG_X86_MCE=y
> # CONFIG_X86_MCE_NONFATAL is not set
> CONFIG_VM86=y
> # CONFIG_TOSHIBA is not set
> # CONFIG_I8K is not set
> # CONFIG_X86_REBOOTFIXUPS is not set
> # CONFIG_MICROCODE is not set
> # CONFIG_X86_MSR is not set
> # CONFIG_X86_CPUID is not set
> # CONFIG_NOHIGHMEM is not set
> CONFIG_HIGHMEM4G=y
> # CONFIG_HIGHMEM64G is not set
> CONFIG_PAGE_OFFSET=0xC0000000
> CONFIG_HIGHMEM=y
> # CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
> CONFIG_NEED_NODE_MEMMAP_SIZE=y
> CONFIG_ARCH_FLATMEM_ENABLE=y
> CONFIG_ARCH_SPARSEMEM_ENABLE=y
> CONFIG_ARCH_SELECT_MEMORY_MODEL=y
> CONFIG_SELECT_MEMORY_MODEL=y
> # CONFIG_FLATMEM_MANUAL is not set
> # CONFIG_DISCONTIGMEM_MANUAL is not set
> CONFIG_SPARSEMEM_MANUAL=y
> CONFIG_SPARSEMEM=y
> CONFIG_HAVE_MEMORY_PRESENT=y
> CONFIG_SPARSEMEM_STATIC=y
> # CONFIG_MEMORY_HOTPLUG is not set
> CONFIG_PAGEFLAGS_EXTENDED=y
> CONFIG_SPLIT_PTLOCK_CPUS=4
> # CONFIG_PHYS_ADDR_T_64BIT is not set
> CONFIG_ZONE_DMA_FLAG=1
> CONFIG_BOUNCE=y
> CONFIG_VIRT_TO_BUS=y
> CONFIG_UNEVICTABLE_LRU=y
> # CONFIG_HIGHPTE is not set
> # CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
> CONFIG_X86_RESERVE_LOW_64K=y
> # CONFIG_MATH_EMULATION is not set
> CONFIG_MTRR=y
> # CONFIG_MTRR_SANITIZER is not set
> # CONFIG_X86_PAT is not set
> # CONFIG_EFI is not set
> CONFIG_SECCOMP=y
> # CONFIG_HZ_100 is not set
> CONFIG_HZ_250=y
> # CONFIG_HZ_300 is not set
> # CONFIG_HZ_1000 is not set
> CONFIG_HZ=250
> # CONFIG_SCHED_HRTICK is not set
> # CONFIG_KEXEC is not set
> # CONFIG_CRASH_DUMP is not set
> CONFIG_PHYSICAL_START=0x100000
> # CONFIG_RELOCATABLE is not set
> CONFIG_PHYSICAL_ALIGN=0x200000
> CONFIG_COMPAT_VDSO=y
> # CONFIG_CMDLINE_BOOL is not set
> CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
> 
> #
> # Power management and ACPI options
> #
> CONFIG_PM=y
> # CONFIG_PM_DEBUG is not set
> CONFIG_PM_SLEEP=y
> CONFIG_SUSPEND=y
> CONFIG_SUSPEND_FREEZER=y
> # CONFIG_HIBERNATION is not set
> CONFIG_ACPI=y
> CONFIG_ACPI_SLEEP=y
> CONFIG_ACPI_PROCFS=y
> CONFIG_ACPI_PROCFS_POWER=y
> CONFIG_ACPI_SYSFS_POWER=y
> CONFIG_ACPI_PROC_EVENT=y
> CONFIG_ACPI_AC=y
> CONFIG_ACPI_BATTERY=y
> CONFIG_ACPI_BUTTON=y
> CONFIG_ACPI_FAN=y
> CONFIG_ACPI_DOCK=y
> CONFIG_ACPI_PROCESSOR=y
> CONFIG_ACPI_THERMAL=y
> # CONFIG_ACPI_CUSTOM_DSDT is not set
> CONFIG_ACPI_BLACKLIST_YEAR=0
> # CONFIG_ACPI_DEBUG is not set
> # CONFIG_ACPI_PCI_SLOT is not set
> CONFIG_X86_PM_TIMER=y
> # CONFIG_ACPI_CONTAINER is not set
> # CONFIG_ACPI_SBS is not set
> # CONFIG_APM is not set
> 
> #
> # CPU Frequency scaling
> #
> CONFIG_CPU_FREQ=y
> CONFIG_CPU_FREQ_TABLE=y
> # CONFIG_CPU_FREQ_DEBUG is not set
> CONFIG_CPU_FREQ_STAT=y
> # CONFIG_CPU_FREQ_STAT_DETAILS is not set
> CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
> # CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
> # CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> # CONFIG_CPU_FREQ_GOV_POWERSAVE is not set
> # CONFIG_CPU_FREQ_GOV_USERSPACE is not set
> # CONFIG_CPU_FREQ_GOV_ONDEMAND is not set
> # CONFIG_CPU_FREQ_GOV_CONSERVATIVE is not set
> 
> #
> # CPUFreq processor drivers
> #
> # CONFIG_X86_ACPI_CPUFREQ is not set
> # CONFIG_X86_POWERNOW_K6 is not set
> # CONFIG_X86_POWERNOW_K7 is not set
> # CONFIG_X86_POWERNOW_K8 is not set
> # CONFIG_X86_GX_SUSPMOD is not set
> CONFIG_X86_SPEEDSTEP_CENTRINO=y
> CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
> # CONFIG_X86_SPEEDSTEP_ICH is not set
> # CONFIG_X86_SPEEDSTEP_SMI is not set
> # CONFIG_X86_P4_CLOCKMOD is not set
> # CONFIG_X86_CPUFREQ_NFORCE2 is not set
> # CONFIG_X86_LONGRUN is not set
> # CONFIG_X86_LONGHAUL is not set
> # CONFIG_X86_E_POWERSAVER is not set
> 
> #
> # shared options
> #
> # CONFIG_X86_SPEEDSTEP_LIB is not set
> CONFIG_CPU_IDLE=y
> CONFIG_CPU_IDLE_GOV_LADDER=y
> 
> #
> # Bus options (PCI etc.)
> #
> CONFIG_PCI=y
> # CONFIG_PCI_GOBIOS is not set
> # CONFIG_PCI_GOMMCONFIG is not set
> # CONFIG_PCI_GODIRECT is not set
> # CONFIG_PCI_GOOLPC is not set
> CONFIG_PCI_GOANY=y
> CONFIG_PCI_BIOS=y
> CONFIG_PCI_DIRECT=y
> CONFIG_PCI_MMCONFIG=y
> CONFIG_PCI_DOMAINS=y
> # CONFIG_PCIEPORTBUS is not set
> # CONFIG_ARCH_SUPPORTS_MSI is not set
> CONFIG_PCI_LEGACY=y
> # CONFIG_PCI_DEBUG is not set
> # CONFIG_PCI_STUB is not set
> CONFIG_ISA_DMA_API=y
> # CONFIG_ISA is not set
> # CONFIG_MCA is not set
> # CONFIG_SCx200 is not set
> # CONFIG_OLPC is not set
> CONFIG_K8_NB=y
> CONFIG_PCCARD=y
> # CONFIG_PCMCIA_DEBUG is not set
> CONFIG_PCMCIA=y
> CONFIG_PCMCIA_LOAD_CIS=y
> CONFIG_PCMCIA_IOCTL=y
> CONFIG_CARDBUS=y
> 
> #
> # PC-card bridges
> #
> CONFIG_YENTA=y
> CONFIG_YENTA_O2=y
> CONFIG_YENTA_RICOH=y
> CONFIG_YENTA_TI=y
> CONFIG_YENTA_ENE_TUNE=y
> CONFIG_YENTA_TOSHIBA=y
> # CONFIG_PD6729 is not set
> # CONFIG_I82092 is not set
> CONFIG_PCCARD_NONSTATIC=y
> CONFIG_HOTPLUG_PCI=y
> # CONFIG_HOTPLUG_PCI_FAKE is not set
> # CONFIG_HOTPLUG_PCI_COMPAQ is not set
> # CONFIG_HOTPLUG_PCI_ACPI is not set
> # CONFIG_HOTPLUG_PCI_CPCI is not set
> CONFIG_HOTPLUG_PCI_SHPC=y
> 
> #
> # Executable file formats / Emulations
> #
> CONFIG_BINFMT_ELF=y
> # CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
> CONFIG_HAVE_AOUT=y
> # CONFIG_BINFMT_AOUT is not set
> CONFIG_BINFMT_MISC=y
> CONFIG_HAVE_ATOMIC_IOMAP=y
> CONFIG_NET=y
> 
> #
> # Networking options
> #
> CONFIG_COMPAT_NET_DEV_OPS=y
> CONFIG_PACKET=y
> CONFIG_PACKET_MMAP=y
> CONFIG_UNIX=y
> CONFIG_XFRM=y
> # CONFIG_XFRM_USER is not set
> # CONFIG_XFRM_SUB_POLICY is not set
> # CONFIG_XFRM_MIGRATE is not set
> # CONFIG_XFRM_STATISTICS is not set
> CONFIG_NET_KEY=y
> # CONFIG_NET_KEY_MIGRATE is not set
> CONFIG_INET=y
> CONFIG_IP_MULTICAST=y
> # CONFIG_IP_ADVANCED_ROUTER is not set
> CONFIG_IP_FIB_HASH=y
> # CONFIG_IP_PNP is not set
> # CONFIG_NET_IPIP is not set
> # CONFIG_NET_IPGRE is not set
> # CONFIG_IP_MROUTE is not set
> # CONFIG_ARPD is not set
> # CONFIG_SYN_COOKIES is not set
> # CONFIG_INET_AH is not set
> # CONFIG_INET_ESP is not set
> # CONFIG_INET_IPCOMP is not set
> # CONFIG_INET_XFRM_TUNNEL is not set
> CONFIG_INET_TUNNEL=y
> CONFIG_INET_XFRM_MODE_TRANSPORT=y
> CONFIG_INET_XFRM_MODE_TUNNEL=y
> CONFIG_INET_XFRM_MODE_BEET=y
> # CONFIG_INET_LRO is not set
> CONFIG_INET_DIAG=y
> CONFIG_INET_TCP_DIAG=y
> # CONFIG_TCP_CONG_ADVANCED is not set
> CONFIG_TCP_CONG_CUBIC=y
> CONFIG_DEFAULT_TCP_CONG="cubic"
> # CONFIG_TCP_MD5SIG is not set
> CONFIG_IPV6=y
> # CONFIG_IPV6_PRIVACY is not set
> # CONFIG_IPV6_ROUTER_PREF is not set
> # CONFIG_IPV6_OPTIMISTIC_DAD is not set
> # CONFIG_INET6_AH is not set
> # CONFIG_INET6_ESP is not set
> # CONFIG_INET6_IPCOMP is not set
> # CONFIG_IPV6_MIP6 is not set
> # CONFIG_INET6_XFRM_TUNNEL is not set
> # CONFIG_INET6_TUNNEL is not set
> CONFIG_INET6_XFRM_MODE_TRANSPORT=y
> CONFIG_INET6_XFRM_MODE_TUNNEL=y
> CONFIG_INET6_XFRM_MODE_BEET=y
> # CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
> CONFIG_IPV6_SIT=y
> CONFIG_IPV6_NDISC_NODETYPE=y
> # CONFIG_IPV6_TUNNEL is not set
> # CONFIG_IPV6_MULTIPLE_TABLES is not set
> # CONFIG_IPV6_MROUTE is not set
> # CONFIG_NETLABEL is not set
> # CONFIG_NETWORK_SECMARK is not set
> CONFIG_NETFILTER=y
> # CONFIG_NETFILTER_DEBUG is not set
> CONFIG_NETFILTER_ADVANCED=y
> 
> #
> # Core Netfilter Configuration
> #
> # CONFIG_NETFILTER_NETLINK_QUEUE is not set
> # CONFIG_NETFILTER_NETLINK_LOG is not set
> # CONFIG_NF_CONNTRACK is not set
> CONFIG_NETFILTER_XTABLES=y
> # CONFIG_NETFILTER_XT_TARGET_CLASSIFY is not set
> # CONFIG_NETFILTER_XT_TARGET_MARK is not set
> # CONFIG_NETFILTER_XT_TARGET_NFLOG is not set
> # CONFIG_NETFILTER_XT_TARGET_NFQUEUE is not set
> # CONFIG_NETFILTER_XT_TARGET_RATEEST is not set
> # CONFIG_NETFILTER_XT_TARGET_TCPMSS is not set
> # CONFIG_NETFILTER_XT_MATCH_COMMENT is not set
> # CONFIG_NETFILTER_XT_MATCH_DCCP is not set
> # CONFIG_NETFILTER_XT_MATCH_DSCP is not set
> # CONFIG_NETFILTER_XT_MATCH_ESP is not set
> # CONFIG_NETFILTER_XT_MATCH_HASHLIMIT is not set
> # CONFIG_NETFILTER_XT_MATCH_IPRANGE is not set
> # CONFIG_NETFILTER_XT_MATCH_LENGTH is not set
> # CONFIG_NETFILTER_XT_MATCH_LIMIT is not set
> # CONFIG_NETFILTER_XT_MATCH_MAC is not set
> # CONFIG_NETFILTER_XT_MATCH_MARK is not set
> # CONFIG_NETFILTER_XT_MATCH_MULTIPORT is not set
> # CONFIG_NETFILTER_XT_MATCH_OWNER is not set
> # CONFIG_NETFILTER_XT_MATCH_POLICY is not set
> # CONFIG_NETFILTER_XT_MATCH_PKTTYPE is not set
> # CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
> # CONFIG_NETFILTER_XT_MATCH_RATEEST is not set
> # CONFIG_NETFILTER_XT_MATCH_REALM is not set
> # CONFIG_NETFILTER_XT_MATCH_RECENT is not set
> # CONFIG_NETFILTER_XT_MATCH_SCTP is not set
> # CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
> # CONFIG_NETFILTER_XT_MATCH_STRING is not set
> # CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
> # CONFIG_NETFILTER_XT_MATCH_TIME is not set
> # CONFIG_NETFILTER_XT_MATCH_U32 is not set
> # CONFIG_IP_VS is not set
> 
> #
> # IP: Netfilter Configuration
> #
> # CONFIG_NF_DEFRAG_IPV4 is not set
> CONFIG_IP_NF_QUEUE=y
> CONFIG_IP_NF_IPTABLES=y
> # CONFIG_IP_NF_MATCH_ADDRTYPE is not set
> # CONFIG_IP_NF_MATCH_AH is not set
> # CONFIG_IP_NF_MATCH_ECN is not set
> # CONFIG_IP_NF_MATCH_TTL is not set
> # CONFIG_IP_NF_FILTER is not set
> # CONFIG_IP_NF_TARGET_LOG is not set
> # CONFIG_IP_NF_TARGET_ULOG is not set
> # CONFIG_IP_NF_MANGLE is not set
> # CONFIG_IP_NF_RAW is not set
> # CONFIG_IP_NF_SECURITY is not set
> # CONFIG_IP_NF_ARPTABLES is not set
> 
> #
> # IPv6: Netfilter Configuration
> #
> # CONFIG_IP6_NF_QUEUE is not set
> # CONFIG_IP6_NF_IPTABLES is not set
> # CONFIG_IP_DCCP is not set
> CONFIG_IP_SCTP=y
> # CONFIG_SCTP_DBG_MSG is not set
> # CONFIG_SCTP_DBG_OBJCNT is not set
> # CONFIG_SCTP_HMAC_NONE is not set
> # CONFIG_SCTP_HMAC_SHA1 is not set
> CONFIG_SCTP_HMAC_MD5=y
> # CONFIG_TIPC is not set
> # CONFIG_ATM is not set
> # CONFIG_BRIDGE is not set
> # CONFIG_NET_DSA is not set
> # CONFIG_VLAN_8021Q is not set
> # CONFIG_DECNET is not set
> # CONFIG_LLC2 is not set
> # CONFIG_IPX is not set
> # CONFIG_ATALK is not set
> # CONFIG_X25 is not set
> # CONFIG_LAPB is not set
> # CONFIG_ECONET is not set
> # CONFIG_WAN_ROUTER is not set
> # CONFIG_NET_SCHED is not set
> # CONFIG_DCB is not set
> 
> #
> # Network testing
> #
> # CONFIG_NET_PKTGEN is not set
> # CONFIG_NET_TCPPROBE is not set
> # CONFIG_HAMRADIO is not set
> # CONFIG_CAN is not set
> # CONFIG_IRDA is not set
> # CONFIG_BT is not set
> # CONFIG_AF_RXRPC is not set
> # CONFIG_PHONET is not set
> CONFIG_WIRELESS=y
> # CONFIG_CFG80211 is not set
> CONFIG_WIRELESS_OLD_REGULATORY=y
> CONFIG_WIRELESS_EXT=y
> CONFIG_WIRELESS_EXT_SYSFS=y
> # CONFIG_LIB80211 is not set
> # CONFIG_MAC80211 is not set
> # CONFIG_WIMAX is not set
> # CONFIG_RFKILL is not set
> # CONFIG_NET_9P is not set
> 
> #
> # Device Drivers
> #
> 
> #
> # Generic Driver Options
> #
> CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
> CONFIG_STANDALONE=y
> CONFIG_PREVENT_FIRMWARE_BUILD=y
> CONFIG_FW_LOADER=y
> CONFIG_FIRMWARE_IN_KERNEL=y
> CONFIG_EXTRA_FIRMWARE=""
> # CONFIG_DEBUG_DRIVER is not set
> # CONFIG_DEBUG_DEVRES is not set
> # CONFIG_SYS_HYPERVISOR is not set
> # CONFIG_CONNECTOR is not set
> # CONFIG_MTD is not set
> CONFIG_PARPORT=y
> CONFIG_PARPORT_PC=y
> # CONFIG_PARPORT_SERIAL is not set
> CONFIG_PARPORT_PC_FIFO=y
> CONFIG_PARPORT_PC_SUPERIO=y
> # CONFIG_PARPORT_PC_PCMCIA is not set
> # CONFIG_PARPORT_GSC is not set
> # CONFIG_PARPORT_AX88796 is not set
> CONFIG_PARPORT_1284=y
> CONFIG_PNP=y
> CONFIG_PNP_DEBUG_MESSAGES=y
> 
> #
> # Protocols
> #
> CONFIG_PNPACPI=y
> CONFIG_BLK_DEV=y
> # CONFIG_BLK_DEV_FD is not set
> # CONFIG_PARIDE is not set
> # CONFIG_BLK_CPQ_DA is not set
> # CONFIG_BLK_CPQ_CISS_DA is not set
> # CONFIG_BLK_DEV_DAC960 is not set
> # CONFIG_BLK_DEV_UMEM is not set
> # CONFIG_BLK_DEV_COW_COMMON is not set
> CONFIG_BLK_DEV_LOOP=y
> # CONFIG_BLK_DEV_CRYPTOLOOP is not set
> # CONFIG_BLK_DEV_NBD is not set
> # CONFIG_BLK_DEV_SX8 is not set
> # CONFIG_BLK_DEV_UB is not set
> # CONFIG_BLK_DEV_RAM is not set
> # CONFIG_CDROM_PKTCDVD is not set
> CONFIG_ATA_OVER_ETH=y
> # CONFIG_VIRTIO_BLK is not set
> # CONFIG_BLK_DEV_HD is not set
> CONFIG_MISC_DEVICES=y
> # CONFIG_IBM_ASM is not set
> # CONFIG_PHANTOM is not set
> # CONFIG_SGI_IOC4 is not set
> # CONFIG_TIFM_CORE is not set
> # CONFIG_ICS932S401 is not set
> # CONFIG_ENCLOSURE_SERVICES is not set
> # CONFIG_HP_ILO is not set
> # CONFIG_C2PORT is not set
> 
> #
> # EEPROM support
> #
> # CONFIG_EEPROM_AT24 is not set
> # CONFIG_EEPROM_LEGACY is not set
> # CONFIG_EEPROM_93CX6 is not set
> CONFIG_HAVE_IDE=y
> # CONFIG_IDE is not set
> 
> #
> # SCSI device support
> #
> # CONFIG_RAID_ATTRS is not set
> CONFIG_SCSI=y
> CONFIG_SCSI_DMA=y
> # CONFIG_SCSI_TGT is not set
> # CONFIG_SCSI_NETLINK is not set
> CONFIG_SCSI_PROC_FS=y
> 
> #
> # SCSI support type (disk, tape, CD-ROM)
> #
> CONFIG_BLK_DEV_SD=y
> # CONFIG_CHR_DEV_ST is not set
> # CONFIG_CHR_DEV_OSST is not set
> CONFIG_BLK_DEV_SR=y
> # CONFIG_BLK_DEV_SR_VENDOR is not set
> CONFIG_CHR_DEV_SG=y
> # CONFIG_CHR_DEV_SCH is not set
> 
> #
> # Some SCSI devices (e.g. CD jukebox) support multiple LUNs
> #
> CONFIG_SCSI_MULTI_LUN=y
> CONFIG_SCSI_CONSTANTS=y
> CONFIG_SCSI_LOGGING=y
> # CONFIG_SCSI_SCAN_ASYNC is not set
> CONFIG_SCSI_WAIT_SCAN=m
> 
> #
> # SCSI Transports
> #
> # CONFIG_SCSI_SPI_ATTRS is not set
> # CONFIG_SCSI_FC_ATTRS is not set
> # CONFIG_SCSI_ISCSI_ATTRS is not set
> # CONFIG_SCSI_SAS_LIBSAS is not set
> # CONFIG_SCSI_SRP_ATTRS is not set
> CONFIG_SCSI_LOWLEVEL=y
> # CONFIG_ISCSI_TCP is not set
> # CONFIG_SCSI_CXGB3_ISCSI is not set
> # CONFIG_BLK_DEV_3W_XXXX_RAID is not set
> # CONFIG_SCSI_3W_9XXX is not set
> # CONFIG_SCSI_ACARD is not set
> # CONFIG_SCSI_AACRAID is not set
> # CONFIG_SCSI_AIC7XXX is not set
> # CONFIG_SCSI_AIC7XXX_OLD is not set
> # CONFIG_SCSI_AIC79XX is not set
> # CONFIG_SCSI_AIC94XX is not set
> # CONFIG_SCSI_DPT_I2O is not set
> # CONFIG_SCSI_ADVANSYS is not set
> # CONFIG_SCSI_ARCMSR is not set
> # CONFIG_MEGARAID_NEWGEN is not set
> # CONFIG_MEGARAID_LEGACY is not set
> # CONFIG_MEGARAID_SAS is not set
> # CONFIG_SCSI_HPTIOP is not set
> # CONFIG_SCSI_BUSLOGIC is not set
> # CONFIG_LIBFC is not set
> # CONFIG_FCOE is not set
> # CONFIG_SCSI_DMX3191D is not set
> # CONFIG_SCSI_EATA is not set
> # CONFIG_SCSI_FUTURE_DOMAIN is not set
> # CONFIG_SCSI_GDTH is not set
> # CONFIG_SCSI_IPS is not set
> # CONFIG_SCSI_INITIO is not set
> # CONFIG_SCSI_INIA100 is not set
> # CONFIG_SCSI_PPA is not set
> # CONFIG_SCSI_IMM is not set
> # CONFIG_SCSI_MVSAS is not set
> # CONFIG_SCSI_STEX is not set
> # CONFIG_SCSI_SYM53C8XX_2 is not set
> # CONFIG_SCSI_IPR is not set
> # CONFIG_SCSI_QLOGIC_1280 is not set
> # CONFIG_SCSI_QLA_FC is not set
> # CONFIG_SCSI_QLA_ISCSI is not set
> # CONFIG_SCSI_LPFC is not set
> # CONFIG_SCSI_DC395x is not set
> # CONFIG_SCSI_DC390T is not set
> # CONFIG_SCSI_NSP32 is not set
> # CONFIG_SCSI_DEBUG is not set
> # CONFIG_SCSI_SRP is not set
> # CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
> # CONFIG_SCSI_DH is not set
> CONFIG_ATA=y
> # CONFIG_ATA_NONSTANDARD is not set
> CONFIG_ATA_ACPI=y
> CONFIG_SATA_PMP=y
> CONFIG_SATA_AHCI=y
> # CONFIG_SATA_SIL24 is not set
> CONFIG_ATA_SFF=y
> # CONFIG_SATA_SVW is not set
> CONFIG_ATA_PIIX=y
> # CONFIG_SATA_MV is not set
> # CONFIG_SATA_NV is not set
> # CONFIG_PDC_ADMA is not set
> # CONFIG_SATA_QSTOR is not set
> # CONFIG_SATA_PROMISE is not set
> # CONFIG_SATA_SX4 is not set
> # CONFIG_SATA_SIL is not set
> # CONFIG_SATA_SIS is not set
> # CONFIG_SATA_ULI is not set
> # CONFIG_SATA_VIA is not set
> # CONFIG_SATA_VITESSE is not set
> # CONFIG_SATA_INIC162X is not set
> # CONFIG_PATA_ACPI is not set
> # CONFIG_PATA_ALI is not set
> # CONFIG_PATA_AMD is not set
> # CONFIG_PATA_ARTOP is not set
> # CONFIG_PATA_ATIIXP is not set
> # CONFIG_PATA_CMD640_PCI is not set
> # CONFIG_PATA_CMD64X is not set
> # CONFIG_PATA_CS5520 is not set
> # CONFIG_PATA_CS5530 is not set
> # CONFIG_PATA_CS5535 is not set
> # CONFIG_PATA_CS5536 is not set
> # CONFIG_PATA_CYPRESS is not set
> # CONFIG_PATA_EFAR is not set
> CONFIG_ATA_GENERIC=y
> # CONFIG_PATA_HPT366 is not set
> # CONFIG_PATA_HPT37X is not set
> # CONFIG_PATA_HPT3X2N is not set
> # CONFIG_PATA_HPT3X3 is not set
> # CONFIG_PATA_IT821X is not set
> # CONFIG_PATA_IT8213 is not set
> # CONFIG_PATA_JMICRON is not set
> # CONFIG_PATA_TRIFLEX is not set
> # CONFIG_PATA_MARVELL is not set
> CONFIG_PATA_MPIIX=y
> # CONFIG_PATA_OLDPIIX is not set
> # CONFIG_PATA_NETCELL is not set
> # CONFIG_PATA_NINJA32 is not set
> # CONFIG_PATA_NS87410 is not set
> # CONFIG_PATA_NS87415 is not set
> # CONFIG_PATA_OPTI is not set
> # CONFIG_PATA_OPTIDMA is not set
> # CONFIG_PATA_PCMCIA is not set
> # CONFIG_PATA_PDC_OLD is not set
> # CONFIG_PATA_RADISYS is not set
> # CONFIG_PATA_RZ1000 is not set
> # CONFIG_PATA_SC1200 is not set
> # CONFIG_PATA_SERVERWORKS is not set
> # CONFIG_PATA_PDC2027X is not set
> # CONFIG_PATA_SIL680 is not set
> # CONFIG_PATA_SIS is not set
> # CONFIG_PATA_VIA is not set
> # CONFIG_PATA_WINBOND is not set
> # CONFIG_PATA_SCH is not set
> # CONFIG_MD is not set
> # CONFIG_FUSION is not set
> 
> #
> # IEEE 1394 (FireWire) support
> #
> 
> #
> # Enable only one of the two stacks, unless you know what you are doing
> #
> # CONFIG_FIREWIRE is not set
> CONFIG_IEEE1394=y
> CONFIG_IEEE1394_OHCI1394=y
> # CONFIG_IEEE1394_PCILYNX is not set
> CONFIG_IEEE1394_SBP2=y
> CONFIG_IEEE1394_SBP2_PHYS_DMA=y
> # CONFIG_IEEE1394_ETH1394_ROM_ENTRY is not set
> # CONFIG_IEEE1394_ETH1394 is not set
> # CONFIG_IEEE1394_RAWIO is not set
> # CONFIG_IEEE1394_VIDEO1394 is not set
> # CONFIG_IEEE1394_DV1394 is not set
> # CONFIG_IEEE1394_VERBOSEDEBUG is not set
> # CONFIG_I2O is not set
> # CONFIG_MACINTOSH_DRIVERS is not set
> CONFIG_NETDEVICES=y
> CONFIG_DUMMY=y
> # CONFIG_BONDING is not set
> # CONFIG_MACVLAN is not set
> # CONFIG_EQUALIZER is not set
> CONFIG_TUN=y
> # CONFIG_VETH is not set
> # CONFIG_NET_SB1000 is not set
> # CONFIG_ARCNET is not set
> # CONFIG_PHYLIB is not set
> CONFIG_NET_ETHERNET=y
> CONFIG_MII=y
> # CONFIG_HAPPYMEAL is not set
> # CONFIG_SUNGEM is not set
> # CONFIG_CASSINI is not set
> CONFIG_NET_VENDOR_3COM=y
> CONFIG_VORTEX=y
> # CONFIG_TYPHOON is not set
> # CONFIG_DNET is not set
> # CONFIG_NET_TULIP is not set
> # CONFIG_HP100 is not set
> # CONFIG_IBM_NEW_EMAC_ZMII is not set
> # CONFIG_IBM_NEW_EMAC_RGMII is not set
> # CONFIG_IBM_NEW_EMAC_TAH is not set
> # CONFIG_IBM_NEW_EMAC_EMAC4 is not set
> # CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
> # CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
> # CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
> CONFIG_NET_PCI=y
> # CONFIG_PCNET32 is not set
> # CONFIG_AMD8111_ETH is not set
> # CONFIG_ADAPTEC_STARFIRE is not set
> # CONFIG_B44 is not set
> # CONFIG_FORCEDETH is not set
> # CONFIG_E100 is not set
> # CONFIG_FEALNX is not set
> # CONFIG_NATSEMI is not set
> # CONFIG_NE2K_PCI is not set
> CONFIG_8139CP=y
> CONFIG_8139TOO=y
> # CONFIG_8139TOO_PIO is not set
> CONFIG_8139TOO_TUNE_TWISTER=y
> CONFIG_8139TOO_8129=y
> # CONFIG_8139_OLD_RX_RESET is not set
> # CONFIG_R6040 is not set
> # CONFIG_SIS900 is not set
> # CONFIG_EPIC100 is not set
> # CONFIG_SMSC9420 is not set
> # CONFIG_SUNDANCE is not set
> # CONFIG_TLAN is not set
> # CONFIG_VIA_RHINE is not set
> # CONFIG_SC92031 is not set
> # CONFIG_NET_POCKET is not set
> # CONFIG_ATL2 is not set
> CONFIG_NETDEV_1000=y
> CONFIG_ACENIC=y
> # CONFIG_ACENIC_OMIT_TIGON_I is not set
> # CONFIG_DL2K is not set
> CONFIG_E1000=y
> # CONFIG_E1000E is not set
> # CONFIG_IP1000 is not set
> # CONFIG_IGB is not set
> # CONFIG_NS83820 is not set
> # CONFIG_HAMACHI is not set
> # CONFIG_YELLOWFIN is not set
> # CONFIG_R8169 is not set
> # CONFIG_SIS190 is not set
> # CONFIG_SKGE is not set
> # CONFIG_SKY2 is not set
> # CONFIG_VIA_VELOCITY is not set
> # CONFIG_TIGON3 is not set
> # CONFIG_BNX2 is not set
> # CONFIG_QLA3XXX is not set
> # CONFIG_ATL1 is not set
> # CONFIG_ATL1E is not set
> # CONFIG_ATL1C is not set
> # CONFIG_JME is not set
> CONFIG_NETDEV_10000=y
> # CONFIG_CHELSIO_T1 is not set
> CONFIG_CHELSIO_T3_DEPENDS=y
> # CONFIG_CHELSIO_T3 is not set
> # CONFIG_ENIC is not set
> # CONFIG_IXGBE is not set
> # CONFIG_IXGB is not set
> # CONFIG_S2IO is not set
> # CONFIG_MYRI10GE is not set
> # CONFIG_NETXEN_NIC is not set
> # CONFIG_NIU is not set
> # CONFIG_MLX4_EN is not set
> # CONFIG_MLX4_CORE is not set
> # CONFIG_TEHUTI is not set
> # CONFIG_BNX2X is not set
> # CONFIG_QLGE is not set
> # CONFIG_SFC is not set
> # CONFIG_BE2NET is not set
> # CONFIG_TR is not set
> 
> #
> # Wireless LAN
> #
> # CONFIG_WLAN_PRE80211 is not set
> # CONFIG_WLAN_80211 is not set
> # CONFIG_IWLWIFI_LEDS is not set
> 
> #
> # Enable WiMAX (Networking options) to see the WiMAX drivers
> #
> 
> #
> # USB Network Adapters
> #
> # CONFIG_USB_CATC is not set
> # CONFIG_USB_KAWETH is not set
> # CONFIG_USB_PEGASUS is not set
> # CONFIG_USB_RTL8150 is not set
> # CONFIG_USB_USBNET is not set
> CONFIG_NET_PCMCIA=y
> # CONFIG_PCMCIA_3C589 is not set
> # CONFIG_PCMCIA_3C574 is not set
> # CONFIG_PCMCIA_FMVJ18X is not set
> # CONFIG_PCMCIA_PCNET is not set
> # CONFIG_PCMCIA_NMCLAN is not set
> # CONFIG_PCMCIA_SMC91C92 is not set
> # CONFIG_PCMCIA_XIRC2PS is not set
> # CONFIG_PCMCIA_AXNET is not set
> # CONFIG_WAN is not set
> # CONFIG_FDDI is not set
> # CONFIG_HIPPI is not set
> # CONFIG_PLIP is not set
> CONFIG_PPP=y
> # CONFIG_PPP_MULTILINK is not set
> # CONFIG_PPP_FILTER is not set
> # CONFIG_PPP_ASYNC is not set
> # CONFIG_PPP_SYNC_TTY is not set
> # CONFIG_PPP_DEFLATE is not set
> # CONFIG_PPP_BSDCOMP is not set
> # CONFIG_PPP_MPPE is not set
> # CONFIG_PPPOE is not set
> # CONFIG_PPPOL2TP is not set
> # CONFIG_SLIP is not set
> CONFIG_SLHC=y
> # CONFIG_NET_FC is not set
> # CONFIG_NETCONSOLE is not set
> # CONFIG_NETPOLL is not set
> # CONFIG_NET_POLL_CONTROLLER is not set
> CONFIG_VIRTIO_NET=y
> # CONFIG_ISDN is not set
> # CONFIG_PHONE is not set
> 
> #
> # Input device support
> #
> CONFIG_INPUT=y
> CONFIG_INPUT_FF_MEMLESS=m
> # CONFIG_INPUT_POLLDEV is not set
> 
> #
> # Userland interfaces
> #
> CONFIG_INPUT_MOUSEDEV=y
> CONFIG_INPUT_MOUSEDEV_PSAUX=y
> CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
> CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
> # CONFIG_INPUT_JOYDEV is not set
> CONFIG_INPUT_EVDEV=y
> # CONFIG_INPUT_EVBUG is not set
> 
> #
> # Input Device Drivers
> #
> CONFIG_INPUT_KEYBOARD=y
> CONFIG_KEYBOARD_ATKBD=y
> # CONFIG_KEYBOARD_SUNKBD is not set
> # CONFIG_KEYBOARD_LKKBD is not set
> # CONFIG_KEYBOARD_XTKBD is not set
> # CONFIG_KEYBOARD_NEWTON is not set
> # CONFIG_KEYBOARD_STOWAWAY is not set
> CONFIG_INPUT_MOUSE=y
> CONFIG_MOUSE_PS2=y
> CONFIG_MOUSE_PS2_ALPS=y
> CONFIG_MOUSE_PS2_LOGIPS2PP=y
> CONFIG_MOUSE_PS2_SYNAPTICS=y
> CONFIG_MOUSE_PS2_LIFEBOOK=y
> CONFIG_MOUSE_PS2_TRACKPOINT=y
> # CONFIG_MOUSE_PS2_ELANTECH is not set
> # CONFIG_MOUSE_PS2_TOUCHKIT is not set
> # CONFIG_MOUSE_SERIAL is not set
> # CONFIG_MOUSE_APPLETOUCH is not set
> # CONFIG_MOUSE_BCM5974 is not set
> # CONFIG_MOUSE_VSXXXAA is not set
> # CONFIG_INPUT_JOYSTICK is not set
> # CONFIG_INPUT_TABLET is not set
> # CONFIG_INPUT_TOUCHSCREEN is not set
> CONFIG_INPUT_MISC=y
> CONFIG_INPUT_PCSPKR=y
> # CONFIG_INPUT_WISTRON_BTNS is not set
> # CONFIG_INPUT_ATLAS_BTNS is not set
> # CONFIG_INPUT_ATI_REMOTE is not set
> # CONFIG_INPUT_ATI_REMOTE2 is not set
> # CONFIG_INPUT_KEYSPAN_REMOTE is not set
> # CONFIG_INPUT_POWERMATE is not set
> # CONFIG_INPUT_YEALINK is not set
> # CONFIG_INPUT_CM109 is not set
> # CONFIG_INPUT_UINPUT is not set
> 
> #
> # Hardware I/O ports
> #
> CONFIG_SERIO=y
> CONFIG_SERIO_I8042=y
> CONFIG_SERIO_SERPORT=y
> # CONFIG_SERIO_CT82C710 is not set
> # CONFIG_SERIO_PARKBD is not set
> # CONFIG_SERIO_PCIPS2 is not set
> CONFIG_SERIO_LIBPS2=y
> CONFIG_SERIO_RAW=y
> # CONFIG_GAMEPORT is not set
> 
> #
> # Character devices
> #
> CONFIG_VT=y
> CONFIG_CONSOLE_TRANSLATIONS=y
> CONFIG_VT_CONSOLE=y
> CONFIG_HW_CONSOLE=y
> # CONFIG_VT_HW_CONSOLE_BINDING is not set
> CONFIG_DEVKMEM=y
> # CONFIG_SERIAL_NONSTANDARD is not set
> # CONFIG_NOZOMI is not set
> 
> #
> # Serial drivers
> #
> CONFIG_SERIAL_8250=y
> CONFIG_SERIAL_8250_CONSOLE=y
> CONFIG_FIX_EARLYCON_MEM=y
> CONFIG_SERIAL_8250_PCI=y
> CONFIG_SERIAL_8250_PNP=y
> # CONFIG_SERIAL_8250_CS is not set
> CONFIG_SERIAL_8250_NR_UARTS=4
> CONFIG_SERIAL_8250_RUNTIME_UARTS=4
> CONFIG_SERIAL_8250_EXTENDED=y
> CONFIG_SERIAL_8250_MANY_PORTS=y
> # CONFIG_SERIAL_8250_SHARE_IRQ is not set
> CONFIG_SERIAL_8250_DETECT_IRQ=y
> # CONFIG_SERIAL_8250_RSA is not set
> 
> #
> # Non-8250 serial port support
> #
> CONFIG_SERIAL_CORE=y
> CONFIG_SERIAL_CORE_CONSOLE=y
> CONFIG_CONSOLE_POLL=y
> # CONFIG_SERIAL_JSM is not set
> CONFIG_UNIX98_PTYS=y
> # CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
> CONFIG_LEGACY_PTYS=y
> CONFIG_LEGACY_PTY_COUNT=256
> # CONFIG_PRINTER is not set
> # CONFIG_PPDEV is not set
> # CONFIG_VIRTIO_CONSOLE is not set
> # CONFIG_IPMI_HANDLER is not set
> # CONFIG_HW_RANDOM is not set
> CONFIG_NVRAM=y
> CONFIG_RTC=y
> # CONFIG_R3964 is not set
> # CONFIG_APPLICOM is not set
> # CONFIG_SONYPI is not set
> 
> #
> # PCMCIA character devices
> #
> # CONFIG_SYNCLINK_CS is not set
> # CONFIG_CARDMAN_4000 is not set
> # CONFIG_CARDMAN_4040 is not set
> # CONFIG_IPWIRELESS is not set
> # CONFIG_MWAVE is not set
> # CONFIG_PC8736x_GPIO is not set
> # CONFIG_NSC_GPIO is not set
> # CONFIG_CS5535_GPIO is not set
> # CONFIG_RAW_DRIVER is not set
> # CONFIG_HPET is not set
> # CONFIG_HANGCHECK_TIMER is not set
> # CONFIG_TCG_TPM is not set
> # CONFIG_TELCLOCK is not set
> CONFIG_DEVPORT=y
> CONFIG_I2C=y
> CONFIG_I2C_BOARDINFO=y
> # CONFIG_I2C_CHARDEV is not set
> CONFIG_I2C_HELPER_AUTO=y
> CONFIG_I2C_ALGOBIT=y
> 
> #
> # I2C Hardware Bus support
> #
> 
> #
> # PC SMBus host controller drivers
> #
> # CONFIG_I2C_ALI1535 is not set
> # CONFIG_I2C_ALI1563 is not set
> # CONFIG_I2C_ALI15X3 is not set
> # CONFIG_I2C_AMD756 is not set
> # CONFIG_I2C_AMD8111 is not set
> CONFIG_I2C_I801=y
> # CONFIG_I2C_ISCH is not set
> # CONFIG_I2C_PIIX4 is not set
> # CONFIG_I2C_NFORCE2 is not set
> # CONFIG_I2C_SIS5595 is not set
> # CONFIG_I2C_SIS630 is not set
> # CONFIG_I2C_SIS96X is not set
> # CONFIG_I2C_VIA is not set
> # CONFIG_I2C_VIAPRO is not set
> 
> #
> # I2C system bus drivers (mostly embedded / system-on-chip)
> #
> # CONFIG_I2C_OCORES is not set
> # CONFIG_I2C_SIMTEC is not set
> 
> #
> # External I2C/SMBus adapter drivers
> #
> # CONFIG_I2C_PARPORT is not set
> # CONFIG_I2C_PARPORT_LIGHT is not set
> # CONFIG_I2C_TAOS_EVM is not set
> # CONFIG_I2C_TINY_USB is not set
> 
> #
> # Graphics adapter I2C/DDC channel drivers
> #
> # CONFIG_I2C_VOODOO3 is not set
> 
> #
> # Other I2C/SMBus bus drivers
> #
> # CONFIG_I2C_PCA_PLATFORM is not set
> # CONFIG_I2C_STUB is not set
> # CONFIG_SCx200_ACB is not set
> 
> #
> # Miscellaneous I2C Chip support
> #
> # CONFIG_DS1682 is not set
> # CONFIG_SENSORS_PCF8574 is not set
> # CONFIG_PCF8575 is not set
> # CONFIG_SENSORS_PCA9539 is not set
> # CONFIG_SENSORS_PCF8591 is not set
> # CONFIG_SENSORS_MAX6875 is not set
> # CONFIG_SENSORS_TSL2550 is not set
> # CONFIG_I2C_DEBUG_CORE is not set
> # CONFIG_I2C_DEBUG_ALGO is not set
> # CONFIG_I2C_DEBUG_BUS is not set
> # CONFIG_I2C_DEBUG_CHIP is not set
> # CONFIG_SPI is not set
> CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
> # CONFIG_GPIOLIB is not set
> # CONFIG_W1 is not set
> CONFIG_POWER_SUPPLY=y
> # CONFIG_POWER_SUPPLY_DEBUG is not set
> # CONFIG_PDA_POWER is not set
> # CONFIG_BATTERY_DS2760 is not set
> # CONFIG_BATTERY_BQ27x00 is not set
> CONFIG_HWMON=y
> # CONFIG_HWMON_VID is not set
> # CONFIG_SENSORS_ABITUGURU is not set
> # CONFIG_SENSORS_ABITUGURU3 is not set
> # CONFIG_SENSORS_AD7414 is not set
> # CONFIG_SENSORS_AD7418 is not set
> # CONFIG_SENSORS_ADM1021 is not set
> # CONFIG_SENSORS_ADM1025 is not set
> # CONFIG_SENSORS_ADM1026 is not set
> # CONFIG_SENSORS_ADM1029 is not set
> # CONFIG_SENSORS_ADM1031 is not set
> # CONFIG_SENSORS_ADM9240 is not set
> # CONFIG_SENSORS_ADT7462 is not set
> # CONFIG_SENSORS_ADT7470 is not set
> # CONFIG_SENSORS_ADT7473 is not set
> # CONFIG_SENSORS_ADT7475 is not set
> # CONFIG_SENSORS_K8TEMP is not set
> # CONFIG_SENSORS_ASB100 is not set
> # CONFIG_SENSORS_ATXP1 is not set
> # CONFIG_SENSORS_DS1621 is not set
> # CONFIG_SENSORS_I5K_AMB is not set
> # CONFIG_SENSORS_F71805F is not set
> # CONFIG_SENSORS_F71882FG is not set
> # CONFIG_SENSORS_F75375S is not set
> # CONFIG_SENSORS_FSCHER is not set
> # CONFIG_SENSORS_FSCPOS is not set
> # CONFIG_SENSORS_FSCHMD is not set
> # CONFIG_SENSORS_GL518SM is not set
> # CONFIG_SENSORS_GL520SM is not set
> # CONFIG_SENSORS_CORETEMP is not set
> # CONFIG_SENSORS_IT87 is not set
> # CONFIG_SENSORS_LM63 is not set
> # CONFIG_SENSORS_LM75 is not set
> # CONFIG_SENSORS_LM77 is not set
> # CONFIG_SENSORS_LM78 is not set
> # CONFIG_SENSORS_LM80 is not set
> # CONFIG_SENSORS_LM83 is not set
> # CONFIG_SENSORS_LM85 is not set
> # CONFIG_SENSORS_LM87 is not set
> # CONFIG_SENSORS_LM90 is not set
> # CONFIG_SENSORS_LM92 is not set
> # CONFIG_SENSORS_LM93 is not set
> # CONFIG_SENSORS_LTC4245 is not set
> # CONFIG_SENSORS_MAX1619 is not set
> # CONFIG_SENSORS_MAX6650 is not set
> # CONFIG_SENSORS_PC87360 is not set
> # CONFIG_SENSORS_PC87427 is not set
> # CONFIG_SENSORS_SIS5595 is not set
> # CONFIG_SENSORS_DME1737 is not set
> # CONFIG_SENSORS_SMSC47M1 is not set
> # CONFIG_SENSORS_SMSC47M192 is not set
> # CONFIG_SENSORS_SMSC47B397 is not set
> # CONFIG_SENSORS_ADS7828 is not set
> # CONFIG_SENSORS_THMC50 is not set
> # CONFIG_SENSORS_VIA686A is not set
> # CONFIG_SENSORS_VT1211 is not set
> # CONFIG_SENSORS_VT8231 is not set
> # CONFIG_SENSORS_W83781D is not set
> # CONFIG_SENSORS_W83791D is not set
> # CONFIG_SENSORS_W83792D is not set
> # CONFIG_SENSORS_W83793 is not set
> # CONFIG_SENSORS_W83L785TS is not set
> # CONFIG_SENSORS_W83L786NG is not set
> # CONFIG_SENSORS_W83627HF is not set
> # CONFIG_SENSORS_W83627EHF is not set
> # CONFIG_SENSORS_HDAPS is not set
> # CONFIG_SENSORS_LIS3LV02D is not set
> # CONFIG_SENSORS_APPLESMC is not set
> # CONFIG_HWMON_DEBUG_CHIP is not set
> CONFIG_THERMAL=y
> # CONFIG_THERMAL_HWMON is not set
> # CONFIG_WATCHDOG is not set
> CONFIG_SSB_POSSIBLE=y
> 
> #
> # Sonics Silicon Backplane
> #
> # CONFIG_SSB is not set
> 
> #
> # Multifunction device drivers
> #
> # CONFIG_MFD_CORE is not set
> # CONFIG_MFD_SM501 is not set
> # CONFIG_HTC_PASIC3 is not set
> # CONFIG_TWL4030_CORE is not set
> # CONFIG_MFD_TMIO is not set
> # CONFIG_PMIC_DA903X is not set
> # CONFIG_MFD_WM8400 is not set
> # CONFIG_MFD_WM8350_I2C is not set
> # CONFIG_MFD_PCF50633 is not set
> # CONFIG_REGULATOR is not set
> 
> #
> # Multimedia devices
> #
> 
> #
> # Multimedia core support
> #
> # CONFIG_VIDEO_DEV is not set
> # CONFIG_DVB_CORE is not set
> # CONFIG_VIDEO_MEDIA is not set
> 
> #
> # Multimedia drivers
> #
> CONFIG_DAB=y
> # CONFIG_USB_DABUSB is not set
> 
> #
> # Graphics support
> #
> CONFIG_AGP=y
> # CONFIG_AGP_ALI is not set
> # CONFIG_AGP_ATI is not set
> # CONFIG_AGP_AMD is not set
> CONFIG_AGP_AMD64=y
> CONFIG_AGP_INTEL=y
> # CONFIG_AGP_NVIDIA is not set
> # CONFIG_AGP_SIS is not set
> # CONFIG_AGP_SWORKS is not set
> # CONFIG_AGP_VIA is not set
> # CONFIG_AGP_EFFICEON is not set
> CONFIG_DRM=y
> # CONFIG_DRM_TDFX is not set
> # CONFIG_DRM_R128 is not set
> CONFIG_DRM_RADEON=y
> CONFIG_DRM_I810=y
> CONFIG_DRM_I830=y
> # CONFIG_DRM_I915 is not set
> # CONFIG_DRM_MGA is not set
> # CONFIG_DRM_SIS is not set
> # CONFIG_DRM_VIA is not set
> # CONFIG_DRM_SAVAGE is not set
> # CONFIG_VGASTATE is not set
> # CONFIG_VIDEO_OUTPUT_CONTROL is not set
> # CONFIG_FB is not set
> # CONFIG_BACKLIGHT_LCD_SUPPORT is not set
> 
> #
> # Display device support
> #
> # CONFIG_DISPLAY_SUPPORT is not set
> 
> #
> # Console display driver support
> #
> CONFIG_VGA_CONSOLE=y
> # CONFIG_VGACON_SOFT_SCROLLBACK is not set
> CONFIG_DUMMY_CONSOLE=y
> CONFIG_SOUND=y
> CONFIG_SOUND_OSS_CORE=y
> CONFIG_SND=y
> CONFIG_SND_TIMER=y
> CONFIG_SND_PCM=y
> CONFIG_SND_RAWMIDI=y
> CONFIG_SND_SEQUENCER=y
> CONFIG_SND_SEQ_DUMMY=y
> CONFIG_SND_OSSEMUL=y
> CONFIG_SND_MIXER_OSS=y
> CONFIG_SND_PCM_OSS=y
> CONFIG_SND_PCM_OSS_PLUGINS=y
> CONFIG_SND_SEQUENCER_OSS=y
> CONFIG_SND_RTCTIMER=y
> CONFIG_SND_SEQ_RTCTIMER_DEFAULT=y
> # CONFIG_SND_DYNAMIC_MINORS is not set
> CONFIG_SND_SUPPORT_OLD_API=y
> CONFIG_SND_VERBOSE_PROCFS=y
> CONFIG_SND_VERBOSE_PRINTK=y
> # CONFIG_SND_DEBUG is not set
> CONFIG_SND_VMASTER=y
> CONFIG_SND_AC97_CODEC=y
> CONFIG_SND_DRIVERS=y
> # CONFIG_SND_DUMMY is not set
> CONFIG_SND_VIRMIDI=y
> # CONFIG_SND_MTPAV is not set
> # CONFIG_SND_MTS64 is not set
> # CONFIG_SND_SERIAL_U16550 is not set
> # CONFIG_SND_MPU401 is not set
> # CONFIG_SND_PORTMAN2X4 is not set
> # CONFIG_SND_AC97_POWER_SAVE is not set
> CONFIG_SND_PCI=y
> # CONFIG_SND_AD1889 is not set
> # CONFIG_SND_ALS300 is not set
> # CONFIG_SND_ALS4000 is not set
> # CONFIG_SND_ALI5451 is not set
> # CONFIG_SND_ATIIXP is not set
> # CONFIG_SND_ATIIXP_MODEM is not set
> # CONFIG_SND_AU8810 is not set
> # CONFIG_SND_AU8820 is not set
> # CONFIG_SND_AU8830 is not set
> # CONFIG_SND_AW2 is not set
> # CONFIG_SND_AZT3328 is not set
> # CONFIG_SND_BT87X is not set
> # CONFIG_SND_CA0106 is not set
> # CONFIG_SND_CMIPCI is not set
> # CONFIG_SND_OXYGEN is not set
> # CONFIG_SND_CS4281 is not set
> # CONFIG_SND_CS46XX is not set
> # CONFIG_SND_CS5530 is not set
> # CONFIG_SND_CS5535AUDIO is not set
> # CONFIG_SND_DARLA20 is not set
> # CONFIG_SND_GINA20 is not set
> # CONFIG_SND_LAYLA20 is not set
> # CONFIG_SND_DARLA24 is not set
> # CONFIG_SND_GINA24 is not set
> # CONFIG_SND_LAYLA24 is not set
> # CONFIG_SND_MONA is not set
> # CONFIG_SND_MIA is not set
> # CONFIG_SND_ECHO3G is not set
> # CONFIG_SND_INDIGO is not set
> # CONFIG_SND_INDIGOIO is not set
> # CONFIG_SND_INDIGODJ is not set
> # CONFIG_SND_EMU10K1 is not set
> # CONFIG_SND_EMU10K1X is not set
> # CONFIG_SND_ENS1370 is not set
> # CONFIG_SND_ENS1371 is not set
> # CONFIG_SND_ES1938 is not set
> # CONFIG_SND_ES1968 is not set
> # CONFIG_SND_FM801 is not set
> # CONFIG_SND_HDA_INTEL is not set
> # CONFIG_SND_HDSP is not set
> # CONFIG_SND_HDSPM is not set
> # CONFIG_SND_HIFIER is not set
> # CONFIG_SND_ICE1712 is not set
> # CONFIG_SND_ICE1724 is not set
> CONFIG_SND_INTEL8X0=y
> # CONFIG_SND_INTEL8X0M is not set
> # CONFIG_SND_KORG1212 is not set
> # CONFIG_SND_MAESTRO3 is not set
> # CONFIG_SND_MIXART is not set
> # CONFIG_SND_NM256 is not set
> # CONFIG_SND_PCXHR is not set
> # CONFIG_SND_RIPTIDE is not set
> # CONFIG_SND_RME32 is not set
> # CONFIG_SND_RME96 is not set
> # CONFIG_SND_RME9652 is not set
> # CONFIG_SND_SIS7019 is not set
> # CONFIG_SND_SONICVIBES is not set
> # CONFIG_SND_TRIDENT is not set
> # CONFIG_SND_VIA82XX is not set
> # CONFIG_SND_VIA82XX_MODEM is not set
> # CONFIG_SND_VIRTUOSO is not set
> # CONFIG_SND_VX222 is not set
> # CONFIG_SND_YMFPCI is not set
> CONFIG_SND_USB=y
> # CONFIG_SND_USB_AUDIO is not set
> # CONFIG_SND_USB_USX2Y is not set
> # CONFIG_SND_USB_CAIAQ is not set
> # CONFIG_SND_USB_US122L is not set
> CONFIG_SND_PCMCIA=y
> # CONFIG_SND_VXPOCKET is not set
> # CONFIG_SND_PDAUDIOCF is not set
> # CONFIG_SND_SOC is not set
> # CONFIG_SOUND_PRIME is not set
> CONFIG_AC97_BUS=y
> CONFIG_HID_SUPPORT=y
> CONFIG_HID=y
> # CONFIG_HID_DEBUG is not set
> # CONFIG_HIDRAW is not set
> 
> #
> # USB Input Devices
> #
> CONFIG_USB_HID=y
> # CONFIG_HID_PID is not set
> # CONFIG_USB_HIDDEV is not set
> 
> #
> # Special HID drivers
> #
> CONFIG_HID_COMPAT=y
> CONFIG_HID_A4TECH=y
> CONFIG_HID_APPLE=y
> CONFIG_HID_BELKIN=y
> CONFIG_HID_CHERRY=y
> CONFIG_HID_CHICONY=y
> CONFIG_HID_CYPRESS=y
> CONFIG_HID_EZKEY=y
> CONFIG_HID_GYRATION=y
> CONFIG_HID_LOGITECH=y
> # CONFIG_LOGITECH_FF is not set
> # CONFIG_LOGIRUMBLEPAD2_FF is not set
> CONFIG_HID_MICROSOFT=y
> CONFIG_HID_MONTEREY=y
> CONFIG_HID_NTRIG=y
> CONFIG_HID_PANTHERLORD=y
> # CONFIG_PANTHERLORD_FF is not set
> CONFIG_HID_PETALYNX=y
> CONFIG_HID_SAMSUNG=y
> CONFIG_HID_SONY=y
> CONFIG_HID_SUNPLUS=y
> # CONFIG_GREENASIA_FF is not set
> CONFIG_HID_TOPSEED=y
> CONFIG_THRUSTMASTER_FF=m
> CONFIG_ZEROPLUS_FF=m
> CONFIG_USB_SUPPORT=y
> CONFIG_USB_ARCH_HAS_HCD=y
> CONFIG_USB_ARCH_HAS_OHCI=y
> CONFIG_USB_ARCH_HAS_EHCI=y
> CONFIG_USB=y
> # CONFIG_USB_DEBUG is not set
> # CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set
> 
> #
> # Miscellaneous USB options
> #
> CONFIG_USB_DEVICEFS=y
> CONFIG_USB_DEVICE_CLASS=y
> CONFIG_USB_DYNAMIC_MINORS=y
> # CONFIG_USB_SUSPEND is not set
> # CONFIG_USB_OTG is not set
> CONFIG_USB_MON=y
> # CONFIG_USB_WUSB is not set
> # CONFIG_USB_WUSB_CBAF is not set
> 
> #
> # USB Host Controller Drivers
> #
> # CONFIG_USB_C67X00_HCD is not set
> CONFIG_USB_EHCI_HCD=y
> # CONFIG_USB_EHCI_ROOT_HUB_TT is not set
> # CONFIG_USB_EHCI_TT_NEWSCHED is not set
> # CONFIG_USB_OXU210HP_HCD is not set
> # CONFIG_USB_ISP116X_HCD is not set
> # CONFIG_USB_ISP1760_HCD is not set
> CONFIG_USB_OHCI_HCD=y
> # CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
> # CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
> CONFIG_USB_OHCI_LITTLE_ENDIAN=y
> CONFIG_USB_UHCI_HCD=y
> # CONFIG_USB_SL811_HCD is not set
> # CONFIG_USB_R8A66597_HCD is not set
> # CONFIG_USB_WHCI_HCD is not set
> # CONFIG_USB_HWA_HCD is not set
> 
> #
> # USB Device Class drivers
> #
> # CONFIG_USB_ACM is not set
> CONFIG_USB_PRINTER=y
> # CONFIG_USB_WDM is not set
> # CONFIG_USB_TMC is not set
> 
> #
> # NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may also be needed;
> #
> 
> #
> # see USB_STORAGE Help for more information
> #
> CONFIG_USB_STORAGE=y
> # CONFIG_USB_STORAGE_DEBUG is not set
> # CONFIG_USB_STORAGE_DATAFAB is not set
> # CONFIG_USB_STORAGE_FREECOM is not set
> # CONFIG_USB_STORAGE_ISD200 is not set
> # CONFIG_USB_STORAGE_USBAT is not set
> # CONFIG_USB_STORAGE_SDDR09 is not set
> # CONFIG_USB_STORAGE_SDDR55 is not set
> # CONFIG_USB_STORAGE_JUMPSHOT is not set
> # CONFIG_USB_STORAGE_ALAUDA is not set
> # CONFIG_USB_STORAGE_ONETOUCH is not set
> # CONFIG_USB_STORAGE_KARMA is not set
> # CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
> # CONFIG_USB_LIBUSUAL is not set
> 
> #
> # USB Imaging devices
> #
> # CONFIG_USB_MDC800 is not set
> # CONFIG_USB_MICROTEK is not set
> 
> #
> # USB port drivers
> #
> # CONFIG_USB_USS720 is not set
> # CONFIG_USB_SERIAL is not set
> 
> #
> # USB Miscellaneous drivers
> #
> # CONFIG_USB_EMI62 is not set
> # CONFIG_USB_EMI26 is not set
> # CONFIG_USB_ADUTUX is not set
> # CONFIG_USB_SEVSEG is not set
> # CONFIG_USB_RIO500 is not set
> # CONFIG_USB_LEGOTOWER is not set
> # CONFIG_USB_LCD is not set
> # CONFIG_USB_BERRY_CHARGE is not set
> # CONFIG_USB_LED is not set
> # CONFIG_USB_CYPRESS_CY7C63 is not set
> # CONFIG_USB_CYTHERM is not set
> # CONFIG_USB_PHIDGET is not set
> # CONFIG_USB_IDMOUSE is not set
> # CONFIG_USB_FTDI_ELAN is not set
> # CONFIG_USB_APPLEDISPLAY is not set
> # CONFIG_USB_SISUSBVGA is not set
> # CONFIG_USB_LD is not set
> # CONFIG_USB_TRANCEVIBRATOR is not set
> # CONFIG_USB_IOWARRIOR is not set
> # CONFIG_USB_TEST is not set
> # CONFIG_USB_ISIGHTFW is not set
> # CONFIG_USB_VST is not set
> # CONFIG_USB_GADGET is not set
> 
> #
> # OTG and related infrastructure
> #
> # CONFIG_UWB is not set
> # CONFIG_MMC is not set
> # CONFIG_MEMSTICK is not set
> # CONFIG_NEW_LEDS is not set
> # CONFIG_ACCESSIBILITY is not set
> CONFIG_INFINIBAND=y
> # CONFIG_INFINIBAND_USER_MAD is not set
> # CONFIG_INFINIBAND_USER_ACCESS is not set
> CONFIG_INFINIBAND_ADDR_TRANS=y
> # CONFIG_INFINIBAND_MTHCA is not set
> # CONFIG_INFINIBAND_AMSO1100 is not set
> # CONFIG_MLX4_INFINIBAND is not set
> # CONFIG_INFINIBAND_NES is not set
> # CONFIG_INFINIBAND_IPOIB is not set
> # CONFIG_INFINIBAND_SRP is not set
> # CONFIG_INFINIBAND_ISER is not set
> CONFIG_EDAC=y
> 
> #
> # Reporting subsystems
> #
> # CONFIG_EDAC_DEBUG is not set
> CONFIG_EDAC_MM_EDAC=y
> # CONFIG_EDAC_AMD76X is not set
> # CONFIG_EDAC_E7XXX is not set
> # CONFIG_EDAC_E752X is not set
> # CONFIG_EDAC_I82875P is not set
> # CONFIG_EDAC_I82975X is not set
> # CONFIG_EDAC_I3000 is not set
> # CONFIG_EDAC_X38 is not set
> # CONFIG_EDAC_I5400 is not set
> # CONFIG_EDAC_I82860 is not set
> # CONFIG_EDAC_R82600 is not set
> # CONFIG_EDAC_I5000 is not set
> # CONFIG_EDAC_I5100 is not set
> # CONFIG_RTC_CLASS is not set
> # CONFIG_DMADEVICES is not set
> # CONFIG_AUXDISPLAY is not set
> # CONFIG_UIO is not set
> # CONFIG_STAGING is not set
> CONFIG_X86_PLATFORM_DEVICES=y
> # CONFIG_ASUS_LAPTOP is not set
> # CONFIG_TC1100_WMI is not set
> # CONFIG_SONY_LAPTOP is not set
> # CONFIG_THINKPAD_ACPI is not set
> # CONFIG_INTEL_MENLOW is not set
> # CONFIG_EEEPC_LAPTOP is not set
> # CONFIG_ACPI_WMI is not set
> # CONFIG_ACPI_ASUS is not set
> # CONFIG_ACPI_TOSHIBA is not set
> 
> #
> # Firmware Drivers
> #
> # CONFIG_EDD is not set
> CONFIG_FIRMWARE_MEMMAP=y
> # CONFIG_DELL_RBU is not set
> # CONFIG_DCDBAS is not set
> CONFIG_DMIID=y
> # CONFIG_ISCSI_IBFT_FIND is not set
> 
> #
> # File systems
> #
> CONFIG_EXT2_FS=y
> CONFIG_EXT2_FS_XATTR=y
> CONFIG_EXT2_FS_POSIX_ACL=y
> CONFIG_EXT2_FS_SECURITY=y
> # CONFIG_EXT2_FS_XIP is not set
> CONFIG_EXT3_FS=y
> CONFIG_EXT3_FS_XATTR=y
> CONFIG_EXT3_FS_POSIX_ACL=y
> CONFIG_EXT3_FS_SECURITY=y
> CONFIG_EXT4_FS=y
> # CONFIG_EXT4DEV_COMPAT is not set
> CONFIG_EXT4_FS_XATTR=y
> CONFIG_EXT4_FS_POSIX_ACL=y
> CONFIG_EXT4_FS_SECURITY=y
> CONFIG_JBD=y
> CONFIG_JBD_DEBUG=y
> CONFIG_JBD2=y
> CONFIG_JBD2_DEBUG=y
> CONFIG_FS_MBCACHE=y
> CONFIG_REISERFS_FS=y
> # CONFIG_REISERFS_CHECK is not set
> # CONFIG_REISERFS_PROC_INFO is not set
> CONFIG_REISERFS_FS_XATTR=y
> CONFIG_REISERFS_FS_POSIX_ACL=y
> # CONFIG_REISERFS_FS_SECURITY is not set
> CONFIG_JFS_FS=y
> CONFIG_JFS_POSIX_ACL=y
> # CONFIG_JFS_SECURITY is not set
> # CONFIG_JFS_DEBUG is not set
> # CONFIG_JFS_STATISTICS is not set
> CONFIG_FS_POSIX_ACL=y
> CONFIG_FILE_LOCKING=y
> CONFIG_XFS_FS=y
> # CONFIG_XFS_QUOTA is not set
> CONFIG_XFS_POSIX_ACL=y
> # CONFIG_XFS_RT is not set
> # CONFIG_XFS_DEBUG is not set
> # CONFIG_GFS2_FS is not set
> CONFIG_OCFS2_FS=y
> CONFIG_OCFS2_FS_O2CB=y
> CONFIG_OCFS2_FS_USERSPACE_CLUSTER=y
> CONFIG_OCFS2_FS_STATS=y
> CONFIG_OCFS2_DEBUG_MASKLOG=y
> # CONFIG_OCFS2_DEBUG_FS is not set
> # CONFIG_OCFS2_FS_POSIX_ACL is not set
> # CONFIG_BTRFS_FS is not set
> CONFIG_DNOTIFY=y
> CONFIG_INOTIFY=y
> CONFIG_INOTIFY_USER=y
> CONFIG_QUOTA=y
> # CONFIG_QUOTA_NETLINK_INTERFACE is not set
> CONFIG_PRINT_QUOTA_WARNING=y
> CONFIG_QUOTA_TREE=y
> # CONFIG_QFMT_V1 is not set
> CONFIG_QFMT_V2=y
> CONFIG_QUOTACTL=y
> # CONFIG_AUTOFS_FS is not set
> # CONFIG_AUTOFS4_FS is not set
> # CONFIG_FUSE_FS is not set
> 
> #
> # CD-ROM/DVD Filesystems
> #
> CONFIG_ISO9660_FS=y
> CONFIG_JOLIET=y
> CONFIG_ZISOFS=y
> CONFIG_UDF_FS=y
> CONFIG_UDF_NLS=y
> 
> #
> # DOS/FAT/NT Filesystems
> #
> CONFIG_FAT_FS=y
> CONFIG_MSDOS_FS=y
> CONFIG_VFAT_FS=y
> CONFIG_FAT_DEFAULT_CODEPAGE=437
> CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
> # CONFIG_NTFS_FS is not set
> 
> #
> # Pseudo filesystems
> #
> CONFIG_PROC_FS=y
> CONFIG_PROC_KCORE=y
> CONFIG_PROC_SYSCTL=y
> CONFIG_PROC_PAGE_MONITOR=y
> CONFIG_SYSFS=y
> CONFIG_TMPFS=y
> # CONFIG_TMPFS_POSIX_ACL is not set
> # CONFIG_HUGETLBFS is not set
> # CONFIG_HUGETLB_PAGE is not set
> CONFIG_CONFIGFS_FS=y
> CONFIG_MISC_FILESYSTEMS=y
> # CONFIG_ADFS_FS is not set
> # CONFIG_AFFS_FS is not set
> # CONFIG_ECRYPT_FS is not set
> # CONFIG_HFS_FS is not set
> # CONFIG_HFSPLUS_FS is not set
> # CONFIG_BEFS_FS is not set
> # CONFIG_BFS_FS is not set
> # CONFIG_EFS_FS is not set
> # CONFIG_CRAMFS is not set
> # CONFIG_SQUASHFS is not set
> # CONFIG_VXFS_FS is not set
> # CONFIG_MINIX_FS is not set
> # CONFIG_OMFS_FS is not set
> # CONFIG_HPFS_FS is not set
> # CONFIG_QNX4FS_FS is not set
> # CONFIG_ROMFS_FS is not set
> # CONFIG_SYSV_FS is not set
> # CONFIG_UFS_FS is not set
> CONFIG_NETWORK_FILESYSTEMS=y
> CONFIG_NFS_FS=y
> CONFIG_NFS_V3=y
> CONFIG_NFS_V3_ACL=y
> CONFIG_NFS_V4=y
> CONFIG_NFSD=y
> CONFIG_NFSD_V2_ACL=y
> CONFIG_NFSD_V3=y
> CONFIG_NFSD_V3_ACL=y
> CONFIG_NFSD_V4=y
> # CONFIG_NFSD_V4_1 is not set
> CONFIG_LOCKD=y
> CONFIG_LOCKD_V4=y
> CONFIG_EXPORTFS=y
> CONFIG_NFS_ACL_SUPPORT=y
> CONFIG_NFS_COMMON=y
> CONFIG_SUNRPC=y
> CONFIG_SUNRPC_GSS=y
> CONFIG_SUNRPC_XPRT_RDMA=y
> # CONFIG_SUNRPC_REGISTER_V4 is not set
> CONFIG_RPCSEC_GSS_KRB5=y
> CONFIG_RPCSEC_GSS_SPKM3=y
> # CONFIG_SMB_FS is not set
> # CONFIG_CIFS is not set
> # CONFIG_NCP_FS is not set
> # CONFIG_CODA_FS is not set
> # CONFIG_AFS_FS is not set
> 
> #
> # Partition Types
> #
> CONFIG_PARTITION_ADVANCED=y
> # CONFIG_ACORN_PARTITION is not set
> # CONFIG_OSF_PARTITION is not set
> # CONFIG_AMIGA_PARTITION is not set
> # CONFIG_ATARI_PARTITION is not set
> # CONFIG_MAC_PARTITION is not set
> CONFIG_MSDOS_PARTITION=y
> # CONFIG_BSD_DISKLABEL is not set
> # CONFIG_MINIX_SUBPARTITION is not set
> # CONFIG_SOLARIS_X86_PARTITION is not set
> # CONFIG_UNIXWARE_DISKLABEL is not set
> # CONFIG_LDM_PARTITION is not set
> # CONFIG_SGI_PARTITION is not set
> # CONFIG_ULTRIX_PARTITION is not set
> # CONFIG_SUN_PARTITION is not set
> # CONFIG_KARMA_PARTITION is not set
> # CONFIG_EFI_PARTITION is not set
> # CONFIG_SYSV68_PARTITION is not set
> CONFIG_NLS=y
> CONFIG_NLS_DEFAULT="iso8859-1"
> CONFIG_NLS_CODEPAGE_437=y
> # CONFIG_NLS_CODEPAGE_737 is not set
> # CONFIG_NLS_CODEPAGE_775 is not set
> # CONFIG_NLS_CODEPAGE_850 is not set
> # CONFIG_NLS_CODEPAGE_852 is not set
> # CONFIG_NLS_CODEPAGE_855 is not set
> # CONFIG_NLS_CODEPAGE_857 is not set
> # CONFIG_NLS_CODEPAGE_860 is not set
> # CONFIG_NLS_CODEPAGE_861 is not set
> # CONFIG_NLS_CODEPAGE_862 is not set
> # CONFIG_NLS_CODEPAGE_863 is not set
> # CONFIG_NLS_CODEPAGE_864 is not set
> # CONFIG_NLS_CODEPAGE_865 is not set
> # CONFIG_NLS_CODEPAGE_866 is not set
> # CONFIG_NLS_CODEPAGE_869 is not set
> # CONFIG_NLS_CODEPAGE_936 is not set
> # CONFIG_NLS_CODEPAGE_950 is not set
> # CONFIG_NLS_CODEPAGE_932 is not set
> # CONFIG_NLS_CODEPAGE_949 is not set
> # CONFIG_NLS_CODEPAGE_874 is not set
> # CONFIG_NLS_ISO8859_8 is not set
> # CONFIG_NLS_CODEPAGE_1250 is not set
> # CONFIG_NLS_CODEPAGE_1251 is not set
> # CONFIG_NLS_ASCII is not set
> CONFIG_NLS_ISO8859_1=y
> # CONFIG_NLS_ISO8859_2 is not set
> # CONFIG_NLS_ISO8859_3 is not set
> # CONFIG_NLS_ISO8859_4 is not set
> # CONFIG_NLS_ISO8859_5 is not set
> # CONFIG_NLS_ISO8859_6 is not set
> # CONFIG_NLS_ISO8859_7 is not set
> # CONFIG_NLS_ISO8859_9 is not set
> # CONFIG_NLS_ISO8859_13 is not set
> # CONFIG_NLS_ISO8859_14 is not set
> # CONFIG_NLS_ISO8859_15 is not set
> # CONFIG_NLS_KOI8_R is not set
> # CONFIG_NLS_KOI8_U is not set
> # CONFIG_NLS_UTF8 is not set
> CONFIG_DLM=y
> CONFIG_DLM_DEBUG=y
> 
> #
> # Kernel hacking
> #
> CONFIG_TRACE_IRQFLAGS_SUPPORT=y
> # CONFIG_PRINTK_TIME is not set
> CONFIG_ENABLE_WARN_DEPRECATED=y
> CONFIG_ENABLE_MUST_CHECK=y
> CONFIG_FRAME_WARN=1024
> CONFIG_MAGIC_SYSRQ=y
> CONFIG_UNUSED_SYMBOLS=y
> CONFIG_DEBUG_FS=y
> # CONFIG_HEADERS_CHECK is not set
> CONFIG_DEBUG_KERNEL=y
> # CONFIG_DEBUG_SHIRQ is not set
> CONFIG_DETECT_SOFTLOCKUP=y
> # CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
> CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
> CONFIG_SCHED_DEBUG=y
> CONFIG_SCHEDSTATS=y
> # CONFIG_TIMER_STATS is not set
> CONFIG_DEBUG_OBJECTS=y
> # CONFIG_DEBUG_OBJECTS_SELFTEST is not set
> CONFIG_DEBUG_OBJECTS_FREE=y
> CONFIG_DEBUG_OBJECTS_TIMERS=y
> CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT=1
> CONFIG_DEBUG_SLAB=y
> CONFIG_DEBUG_SLAB_LEAK=y
> CONFIG_DEBUG_PREEMPT=y
> CONFIG_DEBUG_RT_MUTEXES=y
> CONFIG_DEBUG_PI_LIST=y
> # CONFIG_RT_MUTEX_TESTER is not set
> CONFIG_DEBUG_SPINLOCK=y
> CONFIG_DEBUG_MUTEXES=y
> CONFIG_DEBUG_LOCK_ALLOC=y
> CONFIG_PROVE_LOCKING=y
> CONFIG_LOCKDEP=y
> # CONFIG_LOCK_STAT is not set
> # CONFIG_DEBUG_LOCKDEP is not set
> CONFIG_TRACE_IRQFLAGS=y
> CONFIG_DEBUG_SPINLOCK_SLEEP=y
> # CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
> CONFIG_STACKTRACE=y
> # CONFIG_DEBUG_KOBJECT is not set
> # CONFIG_DEBUG_HIGHMEM is not set
> CONFIG_DEBUG_BUGVERBOSE=y
> CONFIG_DEBUG_INFO=y
> CONFIG_DEBUG_VM=y
> # CONFIG_DEBUG_VIRTUAL is not set
> CONFIG_DEBUG_WRITECOUNT=y
> CONFIG_DEBUG_MEMORY_INIT=y
> CONFIG_DEBUG_LIST=y
> CONFIG_DEBUG_SG=y
> # CONFIG_DEBUG_NOTIFIERS is not set
> CONFIG_ARCH_WANT_FRAME_POINTERS=y
> CONFIG_FRAME_POINTER=y
> # CONFIG_BOOT_PRINTK_DELAY is not set
> # CONFIG_RCU_TORTURE_TEST is not set
> # CONFIG_RCU_CPU_STALL_DETECTOR is not set
> # CONFIG_KPROBES_SANITY_TEST is not set
> # CONFIG_BACKTRACE_SELF_TEST is not set
> # CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
> # CONFIG_LKDTM is not set
> CONFIG_FAULT_INJECTION=y
> CONFIG_FAILSLAB=y
> CONFIG_FAIL_PAGE_ALLOC=y
> CONFIG_FAIL_MAKE_REQUEST=y
> # CONFIG_FAIL_IO_TIMEOUT is not set
> CONFIG_FAULT_INJECTION_DEBUG_FS=y
> # CONFIG_FAULT_INJECTION_STACKTRACE_FILTER is not set
> CONFIG_LATENCYTOP=y
> CONFIG_SYSCTL_SYSCALL_CHECK=y
> CONFIG_USER_STACKTRACE_SUPPORT=y
> CONFIG_NOP_TRACER=y
> CONFIG_HAVE_FUNCTION_TRACER=y
> CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
> CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
> CONFIG_HAVE_DYNAMIC_FTRACE=y
> CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
> CONFIG_HAVE_HW_BRANCH_TRACER=y
> CONFIG_TRACER_MAX_TRACE=y
> CONFIG_RING_BUFFER=y
> CONFIG_TRACING=y
> 
> #
> # Tracers
> #
> CONFIG_FUNCTION_TRACER=y
> CONFIG_FUNCTION_GRAPH_TRACER=y
> # CONFIG_IRQSOFF_TRACER is not set
> # CONFIG_PREEMPT_TRACER is not set
> CONFIG_SYSPROF_TRACER=y
> CONFIG_SCHED_TRACER=y
> CONFIG_CONTEXT_SWITCH_TRACER=y
> # CONFIG_BOOT_TRACER is not set
> # CONFIG_TRACE_BRANCH_PROFILING is not set
> # CONFIG_POWER_TRACER is not set
> CONFIG_STACK_TRACER=y
> # CONFIG_HW_BRANCH_TRACER is not set
> CONFIG_DYNAMIC_FTRACE=y
> CONFIG_FTRACE_MCOUNT_RECORD=y
> # CONFIG_FTRACE_STARTUP_TEST is not set
> # CONFIG_MMIOTRACE is not set
> # CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
> CONFIG_DYNAMIC_PRINTK_DEBUG=y
> # CONFIG_SAMPLES is not set
> CONFIG_HAVE_ARCH_KGDB=y
> CONFIG_KGDB=y
> CONFIG_KGDB_SERIAL_CONSOLE=y
> # CONFIG_KGDB_TESTS is not set
> # CONFIG_STRICT_DEVMEM is not set
> CONFIG_X86_VERBOSE_BOOTUP=y
> CONFIG_EARLY_PRINTK=y
> # CONFIG_EARLY_PRINTK_DBGP is not set
> CONFIG_DEBUG_STACKOVERFLOW=y
> CONFIG_DEBUG_STACK_USAGE=y
> # CONFIG_DEBUG_PAGEALLOC is not set
> # CONFIG_X86_PTDUMP is not set
> CONFIG_DEBUG_RODATA=y
> # CONFIG_DEBUG_RODATA_TEST is not set
> # CONFIG_DEBUG_NX_TEST is not set
> # CONFIG_4KSTACKS is not set
> CONFIG_DOUBLEFAULT=y
> CONFIG_HAVE_MMIOTRACE_SUPPORT=y
> CONFIG_IO_DELAY_TYPE_0X80=0
> CONFIG_IO_DELAY_TYPE_0XED=1
> CONFIG_IO_DELAY_TYPE_UDELAY=2
> CONFIG_IO_DELAY_TYPE_NONE=3
> # CONFIG_IO_DELAY_0X80 is not set
> CONFIG_IO_DELAY_0XED=y
> # CONFIG_IO_DELAY_UDELAY is not set
> # CONFIG_IO_DELAY_NONE is not set
> CONFIG_DEFAULT_IO_DELAY_TYPE=1
> # CONFIG_DEBUG_BOOT_PARAMS is not set
> # CONFIG_CPA_DEBUG is not set
> # CONFIG_OPTIMIZE_INLINING is not set
> 
> #
> # Security options
> #
> CONFIG_KEYS=y
> CONFIG_KEYS_DEBUG_PROC_KEYS=y
> CONFIG_SECURITY=y
> # CONFIG_SECURITYFS is not set
> # CONFIG_SECURITY_NETWORK is not set
> # CONFIG_SECURITY_PATH is not set
> # CONFIG_SECURITY_FILE_CAPABILITIES is not set
> # CONFIG_SECURITY_ROOTPLUG is not set
> CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR=0
> CONFIG_CRYPTO=y
> 
> #
> # Crypto core or helper
> #
> # CONFIG_CRYPTO_FIPS is not set
> CONFIG_CRYPTO_ALGAPI=y
> CONFIG_CRYPTO_ALGAPI2=y
> CONFIG_CRYPTO_AEAD2=y
> CONFIG_CRYPTO_BLKCIPHER=y
> CONFIG_CRYPTO_BLKCIPHER2=y
> CONFIG_CRYPTO_HASH=y
> CONFIG_CRYPTO_HASH2=y
> CONFIG_CRYPTO_RNG2=y
> CONFIG_CRYPTO_MANAGER=y
> CONFIG_CRYPTO_MANAGER2=y
> # CONFIG_CRYPTO_GF128MUL is not set
> # CONFIG_CRYPTO_NULL is not set
> # CONFIG_CRYPTO_CRYPTD is not set
> # CONFIG_CRYPTO_AUTHENC is not set
> # CONFIG_CRYPTO_TEST is not set
> 
> #
> # Authenticated Encryption with Associated Data
> #
> # CONFIG_CRYPTO_CCM is not set
> # CONFIG_CRYPTO_GCM is not set
> # CONFIG_CRYPTO_SEQIV is not set
> 
> #
> # Block modes
> #
> CONFIG_CRYPTO_CBC=y
> # CONFIG_CRYPTO_CTR is not set
> # CONFIG_CRYPTO_CTS is not set
> CONFIG_CRYPTO_ECB=y
> # CONFIG_CRYPTO_LRW is not set
> # CONFIG_CRYPTO_PCBC is not set
> # CONFIG_CRYPTO_XTS is not set
> 
> #
> # Hash modes
> #
> CONFIG_CRYPTO_HMAC=y
> # CONFIG_CRYPTO_XCBC is not set
> 
> #
> # Digest
> #
> CONFIG_CRYPTO_CRC32C=y
> # CONFIG_CRYPTO_CRC32C_INTEL is not set
> # CONFIG_CRYPTO_MD4 is not set
> CONFIG_CRYPTO_MD5=y
> CONFIG_CRYPTO_MICHAEL_MIC=y
> # CONFIG_CRYPTO_RMD128 is not set
> # CONFIG_CRYPTO_RMD160 is not set
> # CONFIG_CRYPTO_RMD256 is not set
> # CONFIG_CRYPTO_RMD320 is not set
> CONFIG_CRYPTO_SHA1=y
> # CONFIG_CRYPTO_SHA256 is not set
> # CONFIG_CRYPTO_SHA512 is not set
> # CONFIG_CRYPTO_TGR192 is not set
> # CONFIG_CRYPTO_WP512 is not set
> 
> #
> # Ciphers
> #
> CONFIG_CRYPTO_AES=y
> # CONFIG_CRYPTO_AES_586 is not set
> # CONFIG_CRYPTO_ANUBIS is not set
> CONFIG_CRYPTO_ARC4=y
> # CONFIG_CRYPTO_BLOWFISH is not set
> # CONFIG_CRYPTO_CAMELLIA is not set
> CONFIG_CRYPTO_CAST5=y
> # CONFIG_CRYPTO_CAST6 is not set
> CONFIG_CRYPTO_DES=y
> # CONFIG_CRYPTO_FCRYPT is not set
> # CONFIG_CRYPTO_KHAZAD is not set
> # CONFIG_CRYPTO_SALSA20 is not set
> # CONFIG_CRYPTO_SALSA20_586 is not set
> # CONFIG_CRYPTO_SEED is not set
> # CONFIG_CRYPTO_SERPENT is not set
> # CONFIG_CRYPTO_TEA is not set
> # CONFIG_CRYPTO_TWOFISH is not set
> # CONFIG_CRYPTO_TWOFISH_586 is not set
> 
> #
> # Compression
> #
> # CONFIG_CRYPTO_DEFLATE is not set
> # CONFIG_CRYPTO_LZO is not set
> 
> #
> # Random Number Generation
> #
> # CONFIG_CRYPTO_ANSI_CPRNG is not set
> CONFIG_CRYPTO_HW=y
> # CONFIG_CRYPTO_DEV_PADLOCK is not set
> # CONFIG_CRYPTO_DEV_GEODE is not set
> # CONFIG_CRYPTO_DEV_HIFN_795X is not set
> CONFIG_HAVE_KVM=y
> CONFIG_VIRTUALIZATION=y
> # CONFIG_KVM is not set
> # CONFIG_LGUEST is not set
> CONFIG_VIRTIO=y
> CONFIG_VIRTIO_RING=y
> CONFIG_VIRTIO_PCI=y
> # CONFIG_VIRTIO_BALLOON is not set
> 
> #
> # Library routines
> #
> CONFIG_BITREVERSE=y
> CONFIG_GENERIC_FIND_FIRST_BIT=y
> CONFIG_GENERIC_FIND_NEXT_BIT=y
> CONFIG_GENERIC_FIND_LAST_BIT=y
> # CONFIG_CRC_CCITT is not set
> CONFIG_CRC16=y
> CONFIG_CRC_T10DIF=y
> CONFIG_CRC_ITU_T=y
> CONFIG_CRC32=y
> # CONFIG_CRC7 is not set
> CONFIG_LIBCRC32C=y
> CONFIG_ZLIB_INFLATE=y
> CONFIG_PLIST=y
> CONFIG_HAS_IOMEM=y
> CONFIG_HAS_IOPORT=y
> CONFIG_HAS_DMA=y

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Mon, Mar 30, 2009 at 10:38:47PM +0300, Benny Halevy wrote:
> On Mar. 30, 2009, 22:07 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Mon, Mar 30, 2009 at 10:03:39PM +0300, Benny Halevy wrote:
> >> On Mar. 30, 2009, 21:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>> On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
> >>>> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> >>>>> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> >>>>>> Hi Bruce,
> >>>>>>
> >>>>>> Here's the latest server patches implementing the NFSv4.1
> >>>>>> Sessions features.
> >>>>>>
> >>>>>> This patchset is based over your for-2.6.30 branch
> >>>>>> and is also available from
> >>>>>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> >>>>> There's a few merge conflicts with my current for-2.6.30--probably my
> >>>>> fault for not pushing that out recently enough, apologies.  Would you
> >>>>> mind updating?
> >>>>>
> >>>>> --b.
> >>>>>
> >>>> Sure. Here's a rebased version with two minor changes (see below)
> >>> On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
> >>> dereference in the callback code.  Looks like it probably happened while
> >>> running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so
> >> Weird. nfs4_xdr_dec_cb_recall+0x4e doesn't seem like a valid IP
> >> for nfsd.ko @a564667.  Bruce, can you please send me you .config file?
> > 
> > I'm building without CONFIG_NFSD_V4_1 for now; .config appended.
> 
> It looks like rpc_res would be NULL without CONFIG_NFSD_V4_1.
> Does the following patch help?

Yep!

--b.

> 
> Benny
> 
> git diff --stat -p
>  fs/nfsd/nfs4callback.c |    8 +++++---
>  1 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
> index 02e0a61..10c6a80 100644
> --- a/fs/nfsd/nfs4callback.c
> +++ b/fs/nfsd/nfs4callback.c
> @@ -441,9 +441,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
>  	status = decode_cb_compound_hdr(&xdr, &hdr);
>  	if (status)
>  		goto out;
> -	status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
> -	if (status)
> -		goto out;
> +	if (rpc_res && rpc_res->res_seq) {
> +		status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
> +		if (status)
> +			goto out;
> +	}
>  	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
>  out:
>  	return status;
> 
> 
> > 
> > Start of my nfs4_xdr_dec_cb_recall, for what it's worth:
> > 
> > c03e4dd0 <nfs4_xdr_dec_cb_recall>:
> > c03e4dd0:       55                      push   %ebp
> > c03e4dd1:       89 e5                   mov    %esp,%ebp
> > c03e4dd3:       83 ec 38                sub    $0x38,%esp
> > c03e4dd6:       89 5d f4                mov    %ebx,-0xc(%ebp)
> > c03e4dd9:       89 75 f8                mov    %esi,-0x8(%ebp)
> > c03e4ddc:       89 7d fc                mov    %edi,-0x4(%ebp)
> > c03e4ddf:       e8 8c ed e1 ff          call   c0203b70 <mcount>
> > c03e4de4:       89 d3                   mov    %edx,%ebx
> > c03e4de6:       89 ce                   mov    %ecx,%esi
> > c03e4de8:       8d 50 2c                lea    0x2c(%eax),%edx
> > c03e4deb:       89 d9                   mov    %ebx,%ecx
> > c03e4ded:       8d 45 e4                lea    -0x1c(%ebp),%eax
> > c03e4df0:       e8 eb c0 43 00          call   c0820ee0 <xdr_init_decode>
> > c03e4df5:       ba 08 00 00 00          mov    $0x8,%edx
> > c03e4dfa:       8d 45 e4                lea    -0x1c(%ebp),%eax
> > c03e4dfd:       e8 0e c1 43 00          call   c0820f10 <xdr_inline_decode>
> > c03e4e02:       85 c0                   test   %eax,%eax
> > c03e4e04:       74 3a                   je     c03e4e40 <nfs4_xdr_dec_cb_recall+0x70>
> > c03e4e06:       8b 50 04                mov    0x4(%eax),%edx
> > c03e4e09:       8d 45 e4                lea    -0x1c(%ebp),%eax
> > c03e4e0c:       0f ca                   bswap  %edx
> > c03e4e0e:       83 c2 04                add    $0x4,%edx
> > c03e4e11:       e8 fa c0 43 00          call   c0820f10 <xdr_inline_decode>
> > c03e4e16:       85 c0                   test   %eax,%eax
> > c03e4e18:       0f 84 8a 00 00 00       je     c03e4ea8 <nfs4_xdr_dec_cb_recall+0xd8>
> > --> c03e4e1e:   8b 06                   mov    (%esi),%eax
> > c03e4e20:       8b 00                   mov    (%eax),%eax
> > c03e4e22:       85 c0                   test   %eax,%eax
> > c03e4e24:       75 32                   jne    c03e4e58 <nfs4_xdr_dec_cb_recall+0x88>
> > c03e4e26:       ba 04 00 00 00          mov    $0x4,%edx
> > c03e4e2b:       8d 45 e4                lea    -0x1c(%ebp),%eax
> > c03e4e2e:       e8 cd fe ff ff          call   c03e4d00 <decode_cb_op_hdr>
> > c03e4e33:       8b 5d f4                mov    -0xc(%ebp),%ebx
> > c03e4e36:       8b 75 f8                mov    -0x8(%ebp),%esi
> > c03e4e39:       8b 7d fc                mov    -0x4(%ebp),%edi
> > c03e4e3c:       89 ec                   mov    %ebp,%esp
> > c03e4e3e:       5d                      pop    %ebp
> > c03e4e3f:       c3                      ret
> > ...
> > 
> > 
> > --b.
> > 
> > #
> > # Automatically generated make config: don't edit
> > # Linux kernel version: 2.6.29-rc8
> > # Mon Mar 30 14:07:10 2009
> > #
> > # CONFIG_64BIT is not set
> > CONFIG_X86_32=y
> > # CONFIG_X86_64 is not set
> > CONFIG_X86=y
> > CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
> > CONFIG_GENERIC_TIME=y
> > CONFIG_GENERIC_CMOS_UPDATE=y
> > CONFIG_CLOCKSOURCE_WATCHDOG=y
> > CONFIG_GENERIC_CLOCKEVENTS=y
> > CONFIG_LOCKDEP_SUPPORT=y
> > CONFIG_STACKTRACE_SUPPORT=y
> > CONFIG_HAVE_LATENCYTOP_SUPPORT=y
> > CONFIG_FAST_CMPXCHG_LOCAL=y
> > CONFIG_MMU=y
> > CONFIG_ZONE_DMA=y
> > CONFIG_GENERIC_ISA_DMA=y
> > CONFIG_GENERIC_IOMAP=y
> > CONFIG_GENERIC_BUG=y
> > CONFIG_GENERIC_HWEIGHT=y
> > CONFIG_ARCH_MAY_HAVE_PC_FDC=y
> > # CONFIG_RWSEM_GENERIC_SPINLOCK is not set
> > CONFIG_RWSEM_XCHGADD_ALGORITHM=y
> > CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
> > CONFIG_GENERIC_CALIBRATE_DELAY=y
> > # CONFIG_GENERIC_TIME_VSYSCALL is not set
> > CONFIG_ARCH_HAS_CPU_RELAX=y
> > CONFIG_ARCH_HAS_DEFAULT_IDLE=y
> > CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
> > # CONFIG_HAVE_SETUP_PER_CPU_AREA is not set
> > # CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set
> > CONFIG_ARCH_HIBERNATION_POSSIBLE=y
> > CONFIG_ARCH_SUSPEND_POSSIBLE=y
> > # CONFIG_ZONE_DMA32 is not set
> > CONFIG_ARCH_POPULATES_NODE_MAP=y
> > # CONFIG_AUDIT_ARCH is not set
> > CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
> > CONFIG_GENERIC_HARDIRQS=y
> > CONFIG_GENERIC_IRQ_PROBE=y
> > CONFIG_X86_BIOS_REBOOT=y
> > CONFIG_KTIME_SCALAR=y
> > CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
> > 
> > #
> > # General setup
> > #
> > CONFIG_EXPERIMENTAL=y
> > CONFIG_BROKEN_ON_SMP=y
> > CONFIG_LOCK_KERNEL=y
> > CONFIG_INIT_ENV_ARG_LIMIT=32
> > CONFIG_LOCALVERSION=""
> > CONFIG_LOCALVERSION_AUTO=y
> > CONFIG_SWAP=y
> > CONFIG_SYSVIPC=y
> > CONFIG_SYSVIPC_SYSCTL=y
> > CONFIG_POSIX_MQUEUE=y
> > CONFIG_BSD_PROCESS_ACCT=y
> > CONFIG_BSD_PROCESS_ACCT_V3=y
> > # CONFIG_TASKSTATS is not set
> > # CONFIG_AUDIT is not set
> > 
> > #
> > # RCU Subsystem
> > #
> > CONFIG_CLASSIC_RCU=y
> > # CONFIG_TREE_RCU is not set
> > # CONFIG_PREEMPT_RCU is not set
> > # CONFIG_TREE_RCU_TRACE is not set
> > # CONFIG_PREEMPT_RCU_TRACE is not set
> > CONFIG_IKCONFIG=y
> > CONFIG_IKCONFIG_PROC=y
> > CONFIG_LOG_BUF_SHIFT=20
> > CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
> > CONFIG_GROUP_SCHED=y
> > CONFIG_FAIR_GROUP_SCHED=y
> > # CONFIG_RT_GROUP_SCHED is not set
> > CONFIG_USER_SCHED=y
> > # CONFIG_CGROUP_SCHED is not set
> > # CONFIG_CGROUPS is not set
> > CONFIG_SYSFS_DEPRECATED=y
> > CONFIG_SYSFS_DEPRECATED_V2=y
> > CONFIG_RELAY=y
> > CONFIG_NAMESPACES=y
> > # CONFIG_UTS_NS is not set
> > # CONFIG_IPC_NS is not set
> > # CONFIG_USER_NS is not set
> > # CONFIG_PID_NS is not set
> > # CONFIG_NET_NS is not set
> > # CONFIG_BLK_DEV_INITRD is not set
> > # CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
> > CONFIG_SYSCTL=y
> > CONFIG_ANON_INODES=y
> > # CONFIG_EMBEDDED is not set
> > CONFIG_UID16=y
> > CONFIG_SYSCTL_SYSCALL=y
> > CONFIG_KALLSYMS=y
> > CONFIG_KALLSYMS_ALL=y
> > # CONFIG_KALLSYMS_EXTRA_PASS is not set
> > CONFIG_HOTPLUG=y
> > CONFIG_PRINTK=y
> > CONFIG_BUG=y
> > CONFIG_ELF_CORE=y
> > CONFIG_PCSPKR_PLATFORM=y
> > CONFIG_BASE_FULL=y
> > CONFIG_FUTEX=y
> > CONFIG_EPOLL=y
> > CONFIG_SIGNALFD=y
> > CONFIG_TIMERFD=y
> > CONFIG_EVENTFD=y
> > CONFIG_SHMEM=y
> > CONFIG_AIO=y
> > CONFIG_VM_EVENT_COUNTERS=y
> > CONFIG_PCI_QUIRKS=y
> > CONFIG_COMPAT_BRK=y
> > CONFIG_SLAB=y
> > # CONFIG_SLUB is not set
> > # CONFIG_SLOB is not set
> > CONFIG_PROFILING=y
> > CONFIG_TRACEPOINTS=y
> > CONFIG_MARKERS=y
> > CONFIG_OPROFILE=y
> > CONFIG_HAVE_OPROFILE=y
> > CONFIG_KPROBES=y
> > CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
> > CONFIG_KRETPROBES=y
> > CONFIG_HAVE_IOREMAP_PROT=y
> > CONFIG_HAVE_KPROBES=y
> > CONFIG_HAVE_KRETPROBES=y
> > CONFIG_HAVE_ARCH_TRACEHOOK=y
> > CONFIG_HAVE_GENERIC_DMA_COHERENT=y
> > CONFIG_SLABINFO=y
> > CONFIG_RT_MUTEXES=y
> > CONFIG_BASE_SMALL=0
> > CONFIG_MODULES=y
> > # CONFIG_MODULE_FORCE_LOAD is not set
> > CONFIG_MODULE_UNLOAD=y
> > # CONFIG_MODULE_FORCE_UNLOAD is not set
> > # CONFIG_MODVERSIONS is not set
> > # CONFIG_MODULE_SRCVERSION_ALL is not set
> > CONFIG_BLOCK=y
> > CONFIG_LBD=y
> > # CONFIG_BLK_DEV_IO_TRACE is not set
> > # CONFIG_BLK_DEV_BSG is not set
> > # CONFIG_BLK_DEV_INTEGRITY is not set
> > 
> > #
> > # IO Schedulers
> > #
> > CONFIG_IOSCHED_NOOP=y
> > CONFIG_IOSCHED_AS=y
> > CONFIG_IOSCHED_DEADLINE=y
> > CONFIG_IOSCHED_CFQ=y
> > CONFIG_DEFAULT_AS=y
> > # CONFIG_DEFAULT_DEADLINE is not set
> > # CONFIG_DEFAULT_CFQ is not set
> > # CONFIG_DEFAULT_NOOP is not set
> > CONFIG_DEFAULT_IOSCHED="anticipatory"
> > CONFIG_FREEZER=y
> > 
> > #
> > # Processor type and features
> > #
> > # CONFIG_NO_HZ is not set
> > # CONFIG_HIGH_RES_TIMERS is not set
> > CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
> > # CONFIG_SMP is not set
> > CONFIG_X86_PC=y
> > # CONFIG_X86_ELAN is not set
> > # CONFIG_X86_VOYAGER is not set
> > # CONFIG_X86_GENERICARCH is not set
> > # CONFIG_X86_VSMP is not set
> > # CONFIG_X86_RDC321X is not set
> > CONFIG_SCHED_OMIT_FRAME_POINTER=y
> > # CONFIG_PARAVIRT_GUEST is not set
> > # CONFIG_MEMTEST is not set
> > # CONFIG_M386 is not set
> > # CONFIG_M486 is not set
> > # CONFIG_M586 is not set
> > # CONFIG_M586TSC is not set
> > # CONFIG_M586MMX is not set
> > CONFIG_M686=y
> > # CONFIG_MPENTIUMII is not set
> > # CONFIG_MPENTIUMIII is not set
> > # CONFIG_MPENTIUMM is not set
> > # CONFIG_MPENTIUM4 is not set
> > # CONFIG_MK6 is not set
> > # CONFIG_MK7 is not set
> > # CONFIG_MK8 is not set
> > # CONFIG_MCRUSOE is not set
> > # CONFIG_MEFFICEON is not set
> > # CONFIG_MWINCHIPC6 is not set
> > # CONFIG_MWINCHIP3D is not set
> > # CONFIG_MGEODEGX1 is not set
> > # CONFIG_MGEODE_LX is not set
> > # CONFIG_MCYRIXIII is not set
> > # CONFIG_MVIAC3_2 is not set
> > # CONFIG_MVIAC7 is not set
> > # CONFIG_MPSC is not set
> > # CONFIG_MCORE2 is not set
> > # CONFIG_GENERIC_CPU is not set
> > # CONFIG_X86_GENERIC is not set
> > CONFIG_X86_CPU=y
> > CONFIG_X86_CMPXCHG=y
> > CONFIG_X86_L1_CACHE_SHIFT=5
> > CONFIG_X86_XADD=y
> > # CONFIG_X86_PPRO_FENCE is not set
> > CONFIG_X86_WP_WORKS_OK=y
> > CONFIG_X86_INVLPG=y
> > CONFIG_X86_BSWAP=y
> > CONFIG_X86_POPAD_OK=y
> > CONFIG_X86_USE_PPRO_CHECKSUM=y
> > CONFIG_X86_TSC=y
> > CONFIG_X86_CMOV=y
> > CONFIG_X86_MINIMUM_CPU_FAMILY=4
> > CONFIG_X86_DEBUGCTLMSR=y
> > CONFIG_CPU_SUP_INTEL=y
> > CONFIG_CPU_SUP_CYRIX_32=y
> > CONFIG_CPU_SUP_AMD=y
> > CONFIG_CPU_SUP_CENTAUR_32=y
> > CONFIG_CPU_SUP_TRANSMETA_32=y
> > CONFIG_CPU_SUP_UMC_32=y
> > CONFIG_X86_DS=y
> > CONFIG_X86_PTRACE_BTS=y
> > CONFIG_HPET_TIMER=y
> > CONFIG_HPET_EMULATE_RTC=y
> > CONFIG_DMI=y
> > # CONFIG_IOMMU_HELPER is not set
> > # CONFIG_IOMMU_API is not set
> > CONFIG_NR_CPUS=1
> > # CONFIG_PREEMPT_NONE is not set
> > # CONFIG_PREEMPT_VOLUNTARY is not set
> > CONFIG_PREEMPT=y
> > # CONFIG_X86_UP_APIC is not set
> > CONFIG_X86_MCE=y
> > # CONFIG_X86_MCE_NONFATAL is not set
> > CONFIG_VM86=y
> > # CONFIG_TOSHIBA is not set
> > # CONFIG_I8K is not set
> > # CONFIG_X86_REBOOTFIXUPS is not set
> > # CONFIG_MICROCODE is not set
> > # CONFIG_X86_MSR is not set
> > # CONFIG_X86_CPUID is not set
> > # CONFIG_NOHIGHMEM is not set
> > CONFIG_HIGHMEM4G=y
> > # CONFIG_HIGHMEM64G is not set
> > CONFIG_PAGE_OFFSET=0xC0000000
> > CONFIG_HIGHMEM=y
> > # CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
> > CONFIG_NEED_NODE_MEMMAP_SIZE=y
> > CONFIG_ARCH_FLATMEM_ENABLE=y
> > CONFIG_ARCH_SPARSEMEM_ENABLE=y
> > CONFIG_ARCH_SELECT_MEMORY_MODEL=y
> > CONFIG_SELECT_MEMORY_MODEL=y
> > # CONFIG_FLATMEM_MANUAL is not set
> > # CONFIG_DISCONTIGMEM_MANUAL is not set
> > CONFIG_SPARSEMEM_MANUAL=y
> > CONFIG_SPARSEMEM=y
> > CONFIG_HAVE_MEMORY_PRESENT=y
> > CONFIG_SPARSEMEM_STATIC=y
> > # CONFIG_MEMORY_HOTPLUG is not set
> > CONFIG_PAGEFLAGS_EXTENDED=y
> > CONFIG_SPLIT_PTLOCK_CPUS=4
> > # CONFIG_PHYS_ADDR_T_64BIT is not set
> > CONFIG_ZONE_DMA_FLAG=1
> > CONFIG_BOUNCE=y
> > CONFIG_VIRT_TO_BUS=y
> > CONFIG_UNEVICTABLE_LRU=y
> > # CONFIG_HIGHPTE is not set
> > # CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
> > CONFIG_X86_RESERVE_LOW_64K=y
> > # CONFIG_MATH_EMULATION is not set
> > CONFIG_MTRR=y
> > # CONFIG_MTRR_SANITIZER is not set
> > # CONFIG_X86_PAT is not set
> > # CONFIG_EFI is not set
> > CONFIG_SECCOMP=y
> > # CONFIG_HZ_100 is not set
> > CONFIG_HZ_250=y
> > # CONFIG_HZ_300 is not set
> > # CONFIG_HZ_1000 is not set
> > CONFIG_HZ=250
> > # CONFIG_SCHED_HRTICK is not set
> > # CONFIG_KEXEC is not set
> > # CONFIG_CRASH_DUMP is not set
> > CONFIG_PHYSICAL_START=0x100000
> > # CONFIG_RELOCATABLE is not set
> > CONFIG_PHYSICAL_ALIGN=0x200000
> > CONFIG_COMPAT_VDSO=y
> > # CONFIG_CMDLINE_BOOL is not set
> > CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
> > 
> > #
> > # Power management and ACPI options
> > #
> > CONFIG_PM=y
> > # CONFIG_PM_DEBUG is not set
> > CONFIG_PM_SLEEP=y
> > CONFIG_SUSPEND=y
> > CONFIG_SUSPEND_FREEZER=y
> > # CONFIG_HIBERNATION is not set
> > CONFIG_ACPI=y
> > CONFIG_ACPI_SLEEP=y
> > CONFIG_ACPI_PROCFS=y
> > CONFIG_ACPI_PROCFS_POWER=y
> > CONFIG_ACPI_SYSFS_POWER=y
> > CONFIG_ACPI_PROC_EVENT=y
> > CONFIG_ACPI_AC=y
> > CONFIG_ACPI_BATTERY=y
> > CONFIG_ACPI_BUTTON=y
> > CONFIG_ACPI_FAN=y
> > CONFIG_ACPI_DOCK=y
> > CONFIG_ACPI_PROCESSOR=y
> > CONFIG_ACPI_THERMAL=y
> > # CONFIG_ACPI_CUSTOM_DSDT is not set
> > CONFIG_ACPI_BLACKLIST_YEAR=0
> > # CONFIG_ACPI_DEBUG is not set
> > # CONFIG_ACPI_PCI_SLOT is not set
> > CONFIG_X86_PM_TIMER=y
> > # CONFIG_ACPI_CONTAINER is not set
> > # CONFIG_ACPI_SBS is not set
> > # CONFIG_APM is not set
> > 
> > #
> > # CPU Frequency scaling
> > #
> > CONFIG_CPU_FREQ=y
> > CONFIG_CPU_FREQ_TABLE=y
> > # CONFIG_CPU_FREQ_DEBUG is not set
> > CONFIG_CPU_FREQ_STAT=y
> > # CONFIG_CPU_FREQ_STAT_DETAILS is not set
> > CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
> > # CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
> > # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
> > # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
> > # CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
> > CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> > # CONFIG_CPU_FREQ_GOV_POWERSAVE is not set
> > # CONFIG_CPU_FREQ_GOV_USERSPACE is not set
> > # CONFIG_CPU_FREQ_GOV_ONDEMAND is not set
> > # CONFIG_CPU_FREQ_GOV_CONSERVATIVE is not set
> > 
> > #
> > # CPUFreq processor drivers
> > #
> > # CONFIG_X86_ACPI_CPUFREQ is not set
> > # CONFIG_X86_POWERNOW_K6 is not set
> > # CONFIG_X86_POWERNOW_K7 is not set
> > # CONFIG_X86_POWERNOW_K8 is not set
> > # CONFIG_X86_GX_SUSPMOD is not set
> > CONFIG_X86_SPEEDSTEP_CENTRINO=y
> > CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
> > # CONFIG_X86_SPEEDSTEP_ICH is not set
> > # CONFIG_X86_SPEEDSTEP_SMI is not set
> > # CONFIG_X86_P4_CLOCKMOD is not set
> > # CONFIG_X86_CPUFREQ_NFORCE2 is not set
> > # CONFIG_X86_LONGRUN is not set
> > # CONFIG_X86_LONGHAUL is not set
> > # CONFIG_X86_E_POWERSAVER is not set
> > 
> > #
> > # shared options
> > #
> > # CONFIG_X86_SPEEDSTEP_LIB is not set
> > CONFIG_CPU_IDLE=y
> > CONFIG_CPU_IDLE_GOV_LADDER=y
> > 
> > #
> > # Bus options (PCI etc.)
> > #
> > CONFIG_PCI=y
> > # CONFIG_PCI_GOBIOS is not set
> > # CONFIG_PCI_GOMMCONFIG is not set
> > # CONFIG_PCI_GODIRECT is not set
> > # CONFIG_PCI_GOOLPC is not set
> > CONFIG_PCI_GOANY=y
> > CONFIG_PCI_BIOS=y
> > CONFIG_PCI_DIRECT=y
> > CONFIG_PCI_MMCONFIG=y
> > CONFIG_PCI_DOMAINS=y
> > # CONFIG_PCIEPORTBUS is not set
> > # CONFIG_ARCH_SUPPORTS_MSI is not set
> > CONFIG_PCI_LEGACY=y
> > # CONFIG_PCI_DEBUG is not set
> > # CONFIG_PCI_STUB is not set
> > CONFIG_ISA_DMA_API=y
> > # CONFIG_ISA is not set
> > # CONFIG_MCA is not set
> > # CONFIG_SCx200 is not set
> > # CONFIG_OLPC is not set
> > CONFIG_K8_NB=y
> > CONFIG_PCCARD=y
> > # CONFIG_PCMCIA_DEBUG is not set
> > CONFIG_PCMCIA=y
> > CONFIG_PCMCIA_LOAD_CIS=y
> > CONFIG_PCMCIA_IOCTL=y
> > CONFIG_CARDBUS=y
> > 
> > #
> > # PC-card bridges
> > #
> > CONFIG_YENTA=y
> > CONFIG_YENTA_O2=y
> > CONFIG_YENTA_RICOH=y
> > CONFIG_YENTA_TI=y
> > CONFIG_YENTA_ENE_TUNE=y
> > CONFIG_YENTA_TOSHIBA=y
> > # CONFIG_PD6729 is not set
> > # CONFIG_I82092 is not set
> > CONFIG_PCCARD_NONSTATIC=y
> > CONFIG_HOTPLUG_PCI=y
> > # CONFIG_HOTPLUG_PCI_FAKE is not set
> > # CONFIG_HOTPLUG_PCI_COMPAQ is not set
> > # CONFIG_HOTPLUG_PCI_ACPI is not set
> > # CONFIG_HOTPLUG_PCI_CPCI is not set
> > CONFIG_HOTPLUG_PCI_SHPC=y
> > 
> > #
> > # Executable file formats / Emulations
> > #
> > CONFIG_BINFMT_ELF=y
> > # CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
> > CONFIG_HAVE_AOUT=y
> > # CONFIG_BINFMT_AOUT is not set
> > CONFIG_BINFMT_MISC=y
> > CONFIG_HAVE_ATOMIC_IOMAP=y
> > CONFIG_NET=y
> > 
> > #
> > # Networking options
> > #
> > CONFIG_COMPAT_NET_DEV_OPS=y
> > CONFIG_PACKET=y
> > CONFIG_PACKET_MMAP=y
> > CONFIG_UNIX=y
> > CONFIG_XFRM=y
> > # CONFIG_XFRM_USER is not set
> > # CONFIG_XFRM_SUB_POLICY is not set
> > # CONFIG_XFRM_MIGRATE is not set
> > # CONFIG_XFRM_STATISTICS is not set
> > CONFIG_NET_KEY=y
> > # CONFIG_NET_KEY_MIGRATE is not set
> > CONFIG_INET=y
> > CONFIG_IP_MULTICAST=y
> > # CONFIG_IP_ADVANCED_ROUTER is not set
> > CONFIG_IP_FIB_HASH=y
> > # CONFIG_IP_PNP is not set
> > # CONFIG_NET_IPIP is not set
> > # CONFIG_NET_IPGRE is not set
> > # CONFIG_IP_MROUTE is not set
> > # CONFIG_ARPD is not set
> > # CONFIG_SYN_COOKIES is not set
> > # CONFIG_INET_AH is not set
> > # CONFIG_INET_ESP is not set
> > # CONFIG_INET_IPCOMP is not set
> > # CONFIG_INET_XFRM_TUNNEL is not set
> > CONFIG_INET_TUNNEL=y
> > CONFIG_INET_XFRM_MODE_TRANSPORT=y
> > CONFIG_INET_XFRM_MODE_TUNNEL=y
> > CONFIG_INET_XFRM_MODE_BEET=y
> > # CONFIG_INET_LRO is not set
> > CONFIG_INET_DIAG=y
> > CONFIG_INET_TCP_DIAG=y
> > # CONFIG_TCP_CONG_ADVANCED is not set
> > CONFIG_TCP_CONG_CUBIC=y
> > CONFIG_DEFAULT_TCP_CONG="cubic"
> > # CONFIG_TCP_MD5SIG is not set
> > CONFIG_IPV6=y
> > # CONFIG_IPV6_PRIVACY is not set
> > # CONFIG_IPV6_ROUTER_PREF is not set
> > # CONFIG_IPV6_OPTIMISTIC_DAD is not set
> > # CONFIG_INET6_AH is not set
> > # CONFIG_INET6_ESP is not set
> > # CONFIG_INET6_IPCOMP is not set
> > # CONFIG_IPV6_MIP6 is not set
> > # CONFIG_INET6_XFRM_TUNNEL is not set
> > # CONFIG_INET6_TUNNEL is not set
> > CONFIG_INET6_XFRM_MODE_TRANSPORT=y
> > CONFIG_INET6_XFRM_MODE_TUNNEL=y
> > CONFIG_INET6_XFRM_MODE_BEET=y
> > # CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
> > CONFIG_IPV6_SIT=y
> > CONFIG_IPV6_NDISC_NODETYPE=y
> > # CONFIG_IPV6_TUNNEL is not set
> > # CONFIG_IPV6_MULTIPLE_TABLES is not set
> > # CONFIG_IPV6_MROUTE is not set
> > # CONFIG_NETLABEL is not set
> > # CONFIG_NETWORK_SECMARK is not set
> > CONFIG_NETFILTER=y
> > # CONFIG_NETFILTER_DEBUG is not set
> > CONFIG_NETFILTER_ADVANCED=y
> > 
> > #
> > # Core Netfilter Configuration
> > #
> > # CONFIG_NETFILTER_NETLINK_QUEUE is not set
> > # CONFIG_NETFILTER_NETLINK_LOG is not set
> > # CONFIG_NF_CONNTRACK is not set
> > CONFIG_NETFILTER_XTABLES=y
> > # CONFIG_NETFILTER_XT_TARGET_CLASSIFY is not set
> > # CONFIG_NETFILTER_XT_TARGET_MARK is not set
> > # CONFIG_NETFILTER_XT_TARGET_NFLOG is not set
> > # CONFIG_NETFILTER_XT_TARGET_NFQUEUE is not set
> > # CONFIG_NETFILTER_XT_TARGET_RATEEST is not set
> > # CONFIG_NETFILTER_XT_TARGET_TCPMSS is not set
> > # CONFIG_NETFILTER_XT_MATCH_COMMENT is not set
> > # CONFIG_NETFILTER_XT_MATCH_DCCP is not set
> > # CONFIG_NETFILTER_XT_MATCH_DSCP is not set
> > # CONFIG_NETFILTER_XT_MATCH_ESP is not set
> > # CONFIG_NETFILTER_XT_MATCH_HASHLIMIT is not set
> > # CONFIG_NETFILTER_XT_MATCH_IPRANGE is not set
> > # CONFIG_NETFILTER_XT_MATCH_LENGTH is not set
> > # CONFIG_NETFILTER_XT_MATCH_LIMIT is not set
> > # CONFIG_NETFILTER_XT_MATCH_MAC is not set
> > # CONFIG_NETFILTER_XT_MATCH_MARK is not set
> > # CONFIG_NETFILTER_XT_MATCH_MULTIPORT is not set
> > # CONFIG_NETFILTER_XT_MATCH_OWNER is not set
> > # CONFIG_NETFILTER_XT_MATCH_POLICY is not set
> > # CONFIG_NETFILTER_XT_MATCH_PKTTYPE is not set
> > # CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
> > # CONFIG_NETFILTER_XT_MATCH_RATEEST is not set
> > # CONFIG_NETFILTER_XT_MATCH_REALM is not set
> > # CONFIG_NETFILTER_XT_MATCH_RECENT is not set
> > # CONFIG_NETFILTER_XT_MATCH_SCTP is not set
> > # CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
> > # CONFIG_NETFILTER_XT_MATCH_STRING is not set
> > # CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
> > # CONFIG_NETFILTER_XT_MATCH_TIME is not set
> > # CONFIG_NETFILTER_XT_MATCH_U32 is not set
> > # CONFIG_IP_VS is not set
> > 
> > #
> > # IP: Netfilter Configuration
> > #
> > # CONFIG_NF_DEFRAG_IPV4 is not set
> > CONFIG_IP_NF_QUEUE=y
> > CONFIG_IP_NF_IPTABLES=y
> > # CONFIG_IP_NF_MATCH_ADDRTYPE is not set
> > # CONFIG_IP_NF_MATCH_AH is not set
> > # CONFIG_IP_NF_MATCH_ECN is not set
> > # CONFIG_IP_NF_MATCH_TTL is not set
> > # CONFIG_IP_NF_FILTER is not set
> > # CONFIG_IP_NF_TARGET_LOG is not set
> > # CONFIG_IP_NF_TARGET_ULOG is not set
> > # CONFIG_IP_NF_MANGLE is not set
> > # CONFIG_IP_NF_RAW is not set
> > # CONFIG_IP_NF_SECURITY is not set
> > # CONFIG_IP_NF_ARPTABLES is not set
> > 
> > #
> > # IPv6: Netfilter Configuration
> > #
> > # CONFIG_IP6_NF_QUEUE is not set
> > # CONFIG_IP6_NF_IPTABLES is not set
> > # CONFIG_IP_DCCP is not set
> > CONFIG_IP_SCTP=y
> > # CONFIG_SCTP_DBG_MSG is not set
> > # CONFIG_SCTP_DBG_OBJCNT is not set
> > # CONFIG_SCTP_HMAC_NONE is not set
> > # CONFIG_SCTP_HMAC_SHA1 is not set
> > CONFIG_SCTP_HMAC_MD5=y
> > # CONFIG_TIPC is not set
> > # CONFIG_ATM is not set
> > # CONFIG_BRIDGE is not set
> > # CONFIG_NET_DSA is not set
> > # CONFIG_VLAN_8021Q is not set
> > # CONFIG_DECNET is not set
> > # CONFIG_LLC2 is not set
> > # CONFIG_IPX is not set
> > # CONFIG_ATALK is not set
> > # CONFIG_X25 is not set
> > # CONFIG_LAPB is not set
> > # CONFIG_ECONET is not set
> > # CONFIG_WAN_ROUTER is not set
> > # CONFIG_NET_SCHED is not set
> > # CONFIG_DCB is not set
> > 
> > #
> > # Network testing
> > #
> > # CONFIG_NET_PKTGEN is not set
> > # CONFIG_NET_TCPPROBE is not set
> > # CONFIG_HAMRADIO is not set
> > # CONFIG_CAN is not set
> > # CONFIG_IRDA is not set
> > # CONFIG_BT is not set
> > # CONFIG_AF_RXRPC is not set
> > # CONFIG_PHONET is not set
> > CONFIG_WIRELESS=y
> > # CONFIG_CFG80211 is not set
> > CONFIG_WIRELESS_OLD_REGULATORY=y
> > CONFIG_WIRELESS_EXT=y
> > CONFIG_WIRELESS_EXT_SYSFS=y
> > # CONFIG_LIB80211 is not set
> > # CONFIG_MAC80211 is not set
> > # CONFIG_WIMAX is not set
> > # CONFIG_RFKILL is not set
> > # CONFIG_NET_9P is not set
> > 
> > #
> > # Device Drivers
> > #
> > 
> > #
> > # Generic Driver Options
> > #
> > CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
> > CONFIG_STANDALONE=y
> > CONFIG_PREVENT_FIRMWARE_BUILD=y
> > CONFIG_FW_LOADER=y
> > CONFIG_FIRMWARE_IN_KERNEL=y
> > CONFIG_EXTRA_FIRMWARE=""
> > # CONFIG_DEBUG_DRIVER is not set
> > # CONFIG_DEBUG_DEVRES is not set
> > # CONFIG_SYS_HYPERVISOR is not set
> > # CONFIG_CONNECTOR is not set
> > # CONFIG_MTD is not set
> > CONFIG_PARPORT=y
> > CONFIG_PARPORT_PC=y
> > # CONFIG_PARPORT_SERIAL is not set
> > CONFIG_PARPORT_PC_FIFO=y
> > CONFIG_PARPORT_PC_SUPERIO=y
> > # CONFIG_PARPORT_PC_PCMCIA is not set
> > # CONFIG_PARPORT_GSC is not set
> > # CONFIG_PARPORT_AX88796 is not set
> > CONFIG_PARPORT_1284=y
> > CONFIG_PNP=y
> > CONFIG_PNP_DEBUG_MESSAGES=y
> > 
> > #
> > # Protocols
> > #
> > CONFIG_PNPACPI=y
> > CONFIG_BLK_DEV=y
> > # CONFIG_BLK_DEV_FD is not set
> > # CONFIG_PARIDE is not set
> > # CONFIG_BLK_CPQ_DA is not set
> > # CONFIG_BLK_CPQ_CISS_DA is not set
> > # CONFIG_BLK_DEV_DAC960 is not set
> > # CONFIG_BLK_DEV_UMEM is not set
> > # CONFIG_BLK_DEV_COW_COMMON is not set
> > CONFIG_BLK_DEV_LOOP=y
> > # CONFIG_BLK_DEV_CRYPTOLOOP is not set
> > # CONFIG_BLK_DEV_NBD is not set
> > # CONFIG_BLK_DEV_SX8 is not set
> > # CONFIG_BLK_DEV_UB is not set
> > # CONFIG_BLK_DEV_RAM is not set
> > # CONFIG_CDROM_PKTCDVD is not set
> > CONFIG_ATA_OVER_ETH=y
> > # CONFIG_VIRTIO_BLK is not set
> > # CONFIG_BLK_DEV_HD is not set
> > CONFIG_MISC_DEVICES=y
> > # CONFIG_IBM_ASM is not set
> > # CONFIG_PHANTOM is not set
> > # CONFIG_SGI_IOC4 is not set
> > # CONFIG_TIFM_CORE is not set
> > # CONFIG_ICS932S401 is not set
> > # CONFIG_ENCLOSURE_SERVICES is not set
> > # CONFIG_HP_ILO is not set
> > # CONFIG_C2PORT is not set
> > 
> > #
> > # EEPROM support
> > #
> > # CONFIG_EEPROM_AT24 is not set
> > # CONFIG_EEPROM_LEGACY is not set
> > # CONFIG_EEPROM_93CX6 is not set
> > CONFIG_HAVE_IDE=y
> > # CONFIG_IDE is not set
> > 
> > #
> > # SCSI device support
> > #
> > # CONFIG_RAID_ATTRS is not set
> > CONFIG_SCSI=y
> > CONFIG_SCSI_DMA=y
> > # CONFIG_SCSI_TGT is not set
> > # CONFIG_SCSI_NETLINK is not set
> > CONFIG_SCSI_PROC_FS=y
> > 
> > #
> > # SCSI support type (disk, tape, CD-ROM)
> > #
> > CONFIG_BLK_DEV_SD=y
> > # CONFIG_CHR_DEV_ST is not set
> > # CONFIG_CHR_DEV_OSST is not set
> > CONFIG_BLK_DEV_SR=y
> > # CONFIG_BLK_DEV_SR_VENDOR is not set
> > CONFIG_CHR_DEV_SG=y
> > # CONFIG_CHR_DEV_SCH is not set
> > 
> > #
> > # Some SCSI devices (e.g. CD jukebox) support multiple LUNs
> > #
> > CONFIG_SCSI_MULTI_LUN=y
> > CONFIG_SCSI_CONSTANTS=y
> > CONFIG_SCSI_LOGGING=y
> > # CONFIG_SCSI_SCAN_ASYNC is not set
> > CONFIG_SCSI_WAIT_SCAN=m
> > 
> > #
> > # SCSI Transports
> > #
> > # CONFIG_SCSI_SPI_ATTRS is not set
> > # CONFIG_SCSI_FC_ATTRS is not set
> > # CONFIG_SCSI_ISCSI_ATTRS is not set
> > # CONFIG_SCSI_SAS_LIBSAS is not set
> > # CONFIG_SCSI_SRP_ATTRS is not set
> > CONFIG_SCSI_LOWLEVEL=y
> > # CONFIG_ISCSI_TCP is not set
> > # CONFIG_SCSI_CXGB3_ISCSI is not set
> > # CONFIG_BLK_DEV_3W_XXXX_RAID is not set
> > # CONFIG_SCSI_3W_9XXX is not set
> > # CONFIG_SCSI_ACARD is not set
> > # CONFIG_SCSI_AACRAID is not set
> > # CONFIG_SCSI_AIC7XXX is not set
> > # CONFIG_SCSI_AIC7XXX_OLD is not set
> > # CONFIG_SCSI_AIC79XX is not set
> > # CONFIG_SCSI_AIC94XX is not set
> > # CONFIG_SCSI_DPT_I2O is not set
> > # CONFIG_SCSI_ADVANSYS is not set
> > # CONFIG_SCSI_ARCMSR is not set
> > # CONFIG_MEGARAID_NEWGEN is not set
> > # CONFIG_MEGARAID_LEGACY is not set
> > # CONFIG_MEGARAID_SAS is not set
> > # CONFIG_SCSI_HPTIOP is not set
> > # CONFIG_SCSI_BUSLOGIC is not set
> > # CONFIG_LIBFC is not set
> > # CONFIG_FCOE is not set
> > # CONFIG_SCSI_DMX3191D is not set
> > # CONFIG_SCSI_EATA is not set
> > # CONFIG_SCSI_FUTURE_DOMAIN is not set
> > # CONFIG_SCSI_GDTH is not set
> > # CONFIG_SCSI_IPS is not set
> > # CONFIG_SCSI_INITIO is not set
> > # CONFIG_SCSI_INIA100 is not set
> > # CONFIG_SCSI_PPA is not set
> > # CONFIG_SCSI_IMM is not set
> > # CONFIG_SCSI_MVSAS is not set
> > # CONFIG_SCSI_STEX is not set
> > # CONFIG_SCSI_SYM53C8XX_2 is not set
> > # CONFIG_SCSI_IPR is not set
> > # CONFIG_SCSI_QLOGIC_1280 is not set
> > # CONFIG_SCSI_QLA_FC is not set
> > # CONFIG_SCSI_QLA_ISCSI is not set
> > # CONFIG_SCSI_LPFC is not set
> > # CONFIG_SCSI_DC395x is not set
> > # CONFIG_SCSI_DC390T is not set
> > # CONFIG_SCSI_NSP32 is not set
> > # CONFIG_SCSI_DEBUG is not set
> > # CONFIG_SCSI_SRP is not set
> > # CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
> > # CONFIG_SCSI_DH is not set
> > CONFIG_ATA=y
> > # CONFIG_ATA_NONSTANDARD is not set
> > CONFIG_ATA_ACPI=y
> > CONFIG_SATA_PMP=y
> > CONFIG_SATA_AHCI=y
> > # CONFIG_SATA_SIL24 is not set
> > CONFIG_ATA_SFF=y
> > # CONFIG_SATA_SVW is not set
> > CONFIG_ATA_PIIX=y
> > # CONFIG_SATA_MV is not set
> > # CONFIG_SATA_NV is not set
> > # CONFIG_PDC_ADMA is not set
> > # CONFIG_SATA_QSTOR is not set
> > # CONFIG_SATA_PROMISE is not set
> > # CONFIG_SATA_SX4 is not set
> > # CONFIG_SATA_SIL is not set
> > # CONFIG_SATA_SIS is not set
> > # CONFIG_SATA_ULI is not set
> > # CONFIG_SATA_VIA is not set
> > # CONFIG_SATA_VITESSE is not set
> > # CONFIG_SATA_INIC162X is not set
> > # CONFIG_PATA_ACPI is not set
> > # CONFIG_PATA_ALI is not set
> > # CONFIG_PATA_AMD is not set
> > # CONFIG_PATA_ARTOP is not set
> > # CONFIG_PATA_ATIIXP is not set
> > # CONFIG_PATA_CMD640_PCI is not set
> > # CONFIG_PATA_CMD64X is not set
> > # CONFIG_PATA_CS5520 is not set
> > # CONFIG_PATA_CS5530 is not set
> > # CONFIG_PATA_CS5535 is not set
> > # CONFIG_PATA_CS5536 is not set
> > # CONFIG_PATA_CYPRESS is not set
> > # CONFIG_PATA_EFAR is not set
> > CONFIG_ATA_GENERIC=y
> > # CONFIG_PATA_HPT366 is not set
> > # CONFIG_PATA_HPT37X is not set
> > # CONFIG_PATA_HPT3X2N is not set
> > # CONFIG_PATA_HPT3X3 is not set
> > # CONFIG_PATA_IT821X is not set
> > # CONFIG_PATA_IT8213 is not set
> > # CONFIG_PATA_JMICRON is not set
> > # CONFIG_PATA_TRIFLEX is not set
> > # CONFIG_PATA_MARVELL is not set
> > CONFIG_PATA_MPIIX=y
> > # CONFIG_PATA_OLDPIIX is not set
> > # CONFIG_PATA_NETCELL is not set
> > # CONFIG_PATA_NINJA32 is not set
> > # CONFIG_PATA_NS87410 is not set
> > # CONFIG_PATA_NS87415 is not set
> > # CONFIG_PATA_OPTI is not set
> > # CONFIG_PATA_OPTIDMA is not set
> > # CONFIG_PATA_PCMCIA is not set
> > # CONFIG_PATA_PDC_OLD is not set
> > # CONFIG_PATA_RADISYS is not set
> > # CONFIG_PATA_RZ1000 is not set
> > # CONFIG_PATA_SC1200 is not set
> > # CONFIG_PATA_SERVERWORKS is not set
> > # CONFIG_PATA_PDC2027X is not set
> > # CONFIG_PATA_SIL680 is not set
> > # CONFIG_PATA_SIS is not set
> > # CONFIG_PATA_VIA is not set
> > # CONFIG_PATA_WINBOND is not set
> > # CONFIG_PATA_SCH is not set
> > # CONFIG_MD is not set
> > # CONFIG_FUSION is not set
> > 
> > #
> > # IEEE 1394 (FireWire) support
> > #
> > 
> > #
> > # Enable only one of the two stacks, unless you know what you are doing
> > #
> > # CONFIG_FIREWIRE is not set
> > CONFIG_IEEE1394=y
> > CONFIG_IEEE1394_OHCI1394=y
> > # CONFIG_IEEE1394_PCILYNX is not set
> > CONFIG_IEEE1394_SBP2=y
> > CONFIG_IEEE1394_SBP2_PHYS_DMA=y
> > # CONFIG_IEEE1394_ETH1394_ROM_ENTRY is not set
> > # CONFIG_IEEE1394_ETH1394 is not set
> > # CONFIG_IEEE1394_RAWIO is not set
> > # CONFIG_IEEE1394_VIDEO1394 is not set
> > # CONFIG_IEEE1394_DV1394 is not set
> > # CONFIG_IEEE1394_VERBOSEDEBUG is not set
> > # CONFIG_I2O is not set
> > # CONFIG_MACINTOSH_DRIVERS is not set
> > CONFIG_NETDEVICES=y
> > CONFIG_DUMMY=y
> > # CONFIG_BONDING is not set
> > # CONFIG_MACVLAN is not set
> > # CONFIG_EQUALIZER is not set
> > CONFIG_TUN=y
> > # CONFIG_VETH is not set
> > # CONFIG_NET_SB1000 is not set
> > # CONFIG_ARCNET is not set
> > # CONFIG_PHYLIB is not set
> > CONFIG_NET_ETHERNET=y
> > CONFIG_MII=y
> > # CONFIG_HAPPYMEAL is not set
> > # CONFIG_SUNGEM is not set
> > # CONFIG_CASSINI is not set
> > CONFIG_NET_VENDOR_3COM=y
> > CONFIG_VORTEX=y
> > # CONFIG_TYPHOON is not set
> > # CONFIG_DNET is not set
> > # CONFIG_NET_TULIP is not set
> > # CONFIG_HP100 is not set
> > # CONFIG_IBM_NEW_EMAC_ZMII is not set
> > # CONFIG_IBM_NEW_EMAC_RGMII is not set
> > # CONFIG_IBM_NEW_EMAC_TAH is not set
> > # CONFIG_IBM_NEW_EMAC_EMAC4 is not set
> > # CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
> > # CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
> > # CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
> > CONFIG_NET_PCI=y
> > # CONFIG_PCNET32 is not set
> > # CONFIG_AMD8111_ETH is not set
> > # CONFIG_ADAPTEC_STARFIRE is not set
> > # CONFIG_B44 is not set
> > # CONFIG_FORCEDETH is not set
> > # CONFIG_E100 is not set
> > # CONFIG_FEALNX is not set
> > # CONFIG_NATSEMI is not set
> > # CONFIG_NE2K_PCI is not set
> > CONFIG_8139CP=y
> > CONFIG_8139TOO=y
> > # CONFIG_8139TOO_PIO is not set
> > CONFIG_8139TOO_TUNE_TWISTER=y
> > CONFIG_8139TOO_8129=y
> > # CONFIG_8139_OLD_RX_RESET is not set
> > # CONFIG_R6040 is not set
> > # CONFIG_SIS900 is not set
> > # CONFIG_EPIC100 is not set
> > # CONFIG_SMSC9420 is not set
> > # CONFIG_SUNDANCE is not set
> > # CONFIG_TLAN is not set
> > # CONFIG_VIA_RHINE is not set
> > # CONFIG_SC92031 is not set
> > # CONFIG_NET_POCKET is not set
> > # CONFIG_ATL2 is not set
> > CONFIG_NETDEV_1000=y
> > CONFIG_ACENIC=y
> > # CONFIG_ACENIC_OMIT_TIGON_I is not set
> > # CONFIG_DL2K is not set
> > CONFIG_E1000=y
> > # CONFIG_E1000E is not set
> > # CONFIG_IP1000 is not set
> > # CONFIG_IGB is not set
> > # CONFIG_NS83820 is not set
> > # CONFIG_HAMACHI is not set
> > # CONFIG_YELLOWFIN is not set
> > # CONFIG_R8169 is not set
> > # CONFIG_SIS190 is not set
> > # CONFIG_SKGE is not set
> > # CONFIG_SKY2 is not set
> > # CONFIG_VIA_VELOCITY is not set
> > # CONFIG_TIGON3 is not set
> > # CONFIG_BNX2 is not set
> > # CONFIG_QLA3XXX is not set
> > # CONFIG_ATL1 is not set
> > # CONFIG_ATL1E is not set
> > # CONFIG_ATL1C is not set
> > # CONFIG_JME is not set
> > CONFIG_NETDEV_10000=y
> > # CONFIG_CHELSIO_T1 is not set
> > CONFIG_CHELSIO_T3_DEPENDS=y
> > # CONFIG_CHELSIO_T3 is not set
> > # CONFIG_ENIC is not set
> > # CONFIG_IXGBE is not set
> > # CONFIG_IXGB is not set
> > # CONFIG_S2IO is not set
> > # CONFIG_MYRI10GE is not set
> > # CONFIG_NETXEN_NIC is not set
> > # CONFIG_NIU is not set
> > # CONFIG_MLX4_EN is not set
> > # CONFIG_MLX4_CORE is not set
> > # CONFIG_TEHUTI is not set
> > # CONFIG_BNX2X is not set
> > # CONFIG_QLGE is not set
> > # CONFIG_SFC is not set
> > # CONFIG_BE2NET is not set
> > # CONFIG_TR is not set
> > 
> > #
> > # Wireless LAN
> > #
> > # CONFIG_WLAN_PRE80211 is not set
> > # CONFIG_WLAN_80211 is not set
> > # CONFIG_IWLWIFI_LEDS is not set
> > 
> > #
> > # Enable WiMAX (Networking options) to see the WiMAX drivers
> > #
> > 
> > #
> > # USB Network Adapters
> > #
> > # CONFIG_USB_CATC is not set
> > # CONFIG_USB_KAWETH is not set
> > # CONFIG_USB_PEGASUS is not set
> > # CONFIG_USB_RTL8150 is not set
> > # CONFIG_USB_USBNET is not set
> > CONFIG_NET_PCMCIA=y
> > # CONFIG_PCMCIA_3C589 is not set
> > # CONFIG_PCMCIA_3C574 is not set
> > # CONFIG_PCMCIA_FMVJ18X is not set
> > # CONFIG_PCMCIA_PCNET is not set
> > # CONFIG_PCMCIA_NMCLAN is not set
> > # CONFIG_PCMCIA_SMC91C92 is not set
> > # CONFIG_PCMCIA_XIRC2PS is not set
> > # CONFIG_PCMCIA_AXNET is not set
> > # CONFIG_WAN is not set
> > # CONFIG_FDDI is not set
> > # CONFIG_HIPPI is not set
> > # CONFIG_PLIP is not set
> > CONFIG_PPP=y
> > # CONFIG_PPP_MULTILINK is not set
> > # CONFIG_PPP_FILTER is not set
> > # CONFIG_PPP_ASYNC is not set
> > # CONFIG_PPP_SYNC_TTY is not set
> > # CONFIG_PPP_DEFLATE is not set
> > # CONFIG_PPP_BSDCOMP is not set
> > # CONFIG_PPP_MPPE is not set
> > # CONFIG_PPPOE is not set
> > # CONFIG_PPPOL2TP is not set
> > # CONFIG_SLIP is not set
> > CONFIG_SLHC=y
> > # CONFIG_NET_FC is not set
> > # CONFIG_NETCONSOLE is not set
> > # CONFIG_NETPOLL is not set
> > # CONFIG_NET_POLL_CONTROLLER is not set
> > CONFIG_VIRTIO_NET=y
> > # CONFIG_ISDN is not set
> > # CONFIG_PHONE is not set
> > 
> > #
> > # Input device support
> > #
> > CONFIG_INPUT=y
> > CONFIG_INPUT_FF_MEMLESS=m
> > # CONFIG_INPUT_POLLDEV is not set
> > 
> > #
> > # Userland interfaces
> > #
> > CONFIG_INPUT_MOUSEDEV=y
> > CONFIG_INPUT_MOUSEDEV_PSAUX=y
> > CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
> > CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
> > # CONFIG_INPUT_JOYDEV is not set
> > CONFIG_INPUT_EVDEV=y
> > # CONFIG_INPUT_EVBUG is not set
> > 
> > #
> > # Input Device Drivers
> > #
> > CONFIG_INPUT_KEYBOARD=y
> > CONFIG_KEYBOARD_ATKBD=y
> > # CONFIG_KEYBOARD_SUNKBD is not set
> > # CONFIG_KEYBOARD_LKKBD is not set
> > # CONFIG_KEYBOARD_XTKBD is not set
> > # CONFIG_KEYBOARD_NEWTON is not set
> > # CONFIG_KEYBOARD_STOWAWAY is not set
> > CONFIG_INPUT_MOUSE=y
> > CONFIG_MOUSE_PS2=y
> > CONFIG_MOUSE_PS2_ALPS=y
> > CONFIG_MOUSE_PS2_LOGIPS2PP=y
> > CONFIG_MOUSE_PS2_SYNAPTICS=y
> > CONFIG_MOUSE_PS2_LIFEBOOK=y
> > CONFIG_MOUSE_PS2_TRACKPOINT=y
> > # CONFIG_MOUSE_PS2_ELANTECH is not set
> > # CONFIG_MOUSE_PS2_TOUCHKIT is not set
> > # CONFIG_MOUSE_SERIAL is not set
> > # CONFIG_MOUSE_APPLETOUCH is not set
> > # CONFIG_MOUSE_BCM5974 is not set
> > # CONFIG_MOUSE_VSXXXAA is not set
> > # CONFIG_INPUT_JOYSTICK is not set
> > # CONFIG_INPUT_TABLET is not set
> > # CONFIG_INPUT_TOUCHSCREEN is not set
> > CONFIG_INPUT_MISC=y
> > CONFIG_INPUT_PCSPKR=y
> > # CONFIG_INPUT_WISTRON_BTNS is not set
> > # CONFIG_INPUT_ATLAS_BTNS is not set
> > # CONFIG_INPUT_ATI_REMOTE is not set
> > # CONFIG_INPUT_ATI_REMOTE2 is not set
> > # CONFIG_INPUT_KEYSPAN_REMOTE is not set
> > # CONFIG_INPUT_POWERMATE is not set
> > # CONFIG_INPUT_YEALINK is not set
> > # CONFIG_INPUT_CM109 is not set
> > # CONFIG_INPUT_UINPUT is not set
> > 
> > #
> > # Hardware I/O ports
> > #
> > CONFIG_SERIO=y
> > CONFIG_SERIO_I8042=y
> > CONFIG_SERIO_SERPORT=y
> > # CONFIG_SERIO_CT82C710 is not set
> > # CONFIG_SERIO_PARKBD is not set
> > # CONFIG_SERIO_PCIPS2 is not set
> > CONFIG_SERIO_LIBPS2=y
> > CONFIG_SERIO_RAW=y
> > # CONFIG_GAMEPORT is not set
> > 
> > #
> > # Character devices
> > #
> > CONFIG_VT=y
> > CONFIG_CONSOLE_TRANSLATIONS=y
> > CONFIG_VT_CONSOLE=y
> > CONFIG_HW_CONSOLE=y
> > # CONFIG_VT_HW_CONSOLE_BINDING is not set
> > CONFIG_DEVKMEM=y
> > # CONFIG_SERIAL_NONSTANDARD is not set
> > # CONFIG_NOZOMI is not set
> > 
> > #
> > # Serial drivers
> > #
> > CONFIG_SERIAL_8250=y
> > CONFIG_SERIAL_8250_CONSOLE=y
> > CONFIG_FIX_EARLYCON_MEM=y
> > CONFIG_SERIAL_8250_PCI=y
> > CONFIG_SERIAL_8250_PNP=y
> > # CONFIG_SERIAL_8250_CS is not set
> > CONFIG_SERIAL_8250_NR_UARTS=4
> > CONFIG_SERIAL_8250_RUNTIME_UARTS=4
> > CONFIG_SERIAL_8250_EXTENDED=y
> > CONFIG_SERIAL_8250_MANY_PORTS=y
> > # CONFIG_SERIAL_8250_SHARE_IRQ is not set
> > CONFIG_SERIAL_8250_DETECT_IRQ=y
> > # CONFIG_SERIAL_8250_RSA is not set
> > 
> > #
> > # Non-8250 serial port support
> > #
> > CONFIG_SERIAL_CORE=y
> > CONFIG_SERIAL_CORE_CONSOLE=y
> > CONFIG_CONSOLE_POLL=y
> > # CONFIG_SERIAL_JSM is not set
> > CONFIG_UNIX98_PTYS=y
> > # CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
> > CONFIG_LEGACY_PTYS=y
> > CONFIG_LEGACY_PTY_COUNT=256
> > # CONFIG_PRINTER is not set
> > # CONFIG_PPDEV is not set
> > # CONFIG_VIRTIO_CONSOLE is not set
> > # CONFIG_IPMI_HANDLER is not set
> > # CONFIG_HW_RANDOM is not set
> > CONFIG_NVRAM=y
> > CONFIG_RTC=y
> > # CONFIG_R3964 is not set
> > # CONFIG_APPLICOM is not set
> > # CONFIG_SONYPI is not set
> > 
> > #
> > # PCMCIA character devices
> > #
> > # CONFIG_SYNCLINK_CS is not set
> > # CONFIG_CARDMAN_4000 is not set
> > # CONFIG_CARDMAN_4040 is not set
> > # CONFIG_IPWIRELESS is not set
> > # CONFIG_MWAVE is not set
> > # CONFIG_PC8736x_GPIO is not set
> > # CONFIG_NSC_GPIO is not set
> > # CONFIG_CS5535_GPIO is not set
> > # CONFIG_RAW_DRIVER is not set
> > # CONFIG_HPET is not set
> > # CONFIG_HANGCHECK_TIMER is not set
> > # CONFIG_TCG_TPM is not set
> > # CONFIG_TELCLOCK is not set
> > CONFIG_DEVPORT=y
> > CONFIG_I2C=y
> > CONFIG_I2C_BOARDINFO=y
> > # CONFIG_I2C_CHARDEV is not set
> > CONFIG_I2C_HELPER_AUTO=y
> > CONFIG_I2C_ALGOBIT=y
> > 
> > #
> > # I2C Hardware Bus support
> > #
> > 
> > #
> > # PC SMBus host controller drivers
> > #
> > # CONFIG_I2C_ALI1535 is not set
> > # CONFIG_I2C_ALI1563 is not set
> > # CONFIG_I2C_ALI15X3 is not set
> > # CONFIG_I2C_AMD756 is not set
> > # CONFIG_I2C_AMD8111 is not set
> > CONFIG_I2C_I801=y
> > # CONFIG_I2C_ISCH is not set
> > # CONFIG_I2C_PIIX4 is not set
> > # CONFIG_I2C_NFORCE2 is not set
> > # CONFIG_I2C_SIS5595 is not set
> > # CONFIG_I2C_SIS630 is not set
> > # CONFIG_I2C_SIS96X is not set
> > # CONFIG_I2C_VIA is not set
> > # CONFIG_I2C_VIAPRO is not set
> > 
> > #
> > # I2C system bus drivers (mostly embedded / system-on-chip)
> > #
> > # CONFIG_I2C_OCORES is not set
> > # CONFIG_I2C_SIMTEC is not set
> > 
> > #
> > # External I2C/SMBus adapter drivers
> > #
> > # CONFIG_I2C_PARPORT is not set
> > # CONFIG_I2C_PARPORT_LIGHT is not set
> > # CONFIG_I2C_TAOS_EVM is not set
> > # CONFIG_I2C_TINY_USB is not set
> > 
> > #
> > # Graphics adapter I2C/DDC channel drivers
> > #
> > # CONFIG_I2C_VOODOO3 is not set
> > 
> > #
> > # Other I2C/SMBus bus drivers
> > #
> > # CONFIG_I2C_PCA_PLATFORM is not set
> > # CONFIG_I2C_STUB is not set
> > # CONFIG_SCx200_ACB is not set
> > 
> > #
> > # Miscellaneous I2C Chip support
> > #
> > # CONFIG_DS1682 is not set
> > # CONFIG_SENSORS_PCF8574 is not set
> > # CONFIG_PCF8575 is not set
> > # CONFIG_SENSORS_PCA9539 is not set
> > # CONFIG_SENSORS_PCF8591 is not set
> > # CONFIG_SENSORS_MAX6875 is not set
> > # CONFIG_SENSORS_TSL2550 is not set
> > # CONFIG_I2C_DEBUG_CORE is not set
> > # CONFIG_I2C_DEBUG_ALGO is not set
> > # CONFIG_I2C_DEBUG_BUS is not set
> > # CONFIG_I2C_DEBUG_CHIP is not set
> > # CONFIG_SPI is not set
> > CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
> > # CONFIG_GPIOLIB is not set
> > # CONFIG_W1 is not set
> > CONFIG_POWER_SUPPLY=y
> > # CONFIG_POWER_SUPPLY_DEBUG is not set
> > # CONFIG_PDA_POWER is not set
> > # CONFIG_BATTERY_DS2760 is not set
> > # CONFIG_BATTERY_BQ27x00 is not set
> > CONFIG_HWMON=y
> > # CONFIG_HWMON_VID is not set
> > # CONFIG_SENSORS_ABITUGURU is not set
> > # CONFIG_SENSORS_ABITUGURU3 is not set
> > # CONFIG_SENSORS_AD7414 is not set
> > # CONFIG_SENSORS_AD7418 is not set
> > # CONFIG_SENSORS_ADM1021 is not set
> > # CONFIG_SENSORS_ADM1025 is not set
> > # CONFIG_SENSORS_ADM1026 is not set
> > # CONFIG_SENSORS_ADM1029 is not set
> > # CONFIG_SENSORS_ADM1031 is not set
> > # CONFIG_SENSORS_ADM9240 is not set
> > # CONFIG_SENSORS_ADT7462 is not set
> > # CONFIG_SENSORS_ADT7470 is not set
> > # CONFIG_SENSORS_ADT7473 is not set
> > # CONFIG_SENSORS_ADT7475 is not set
> > # CONFIG_SENSORS_K8TEMP is not set
> > # CONFIG_SENSORS_ASB100 is not set
> > # CONFIG_SENSORS_ATXP1 is not set
> > # CONFIG_SENSORS_DS1621 is not set
> > # CONFIG_SENSORS_I5K_AMB is not set
> > # CONFIG_SENSORS_F71805F is not set
> > # CONFIG_SENSORS_F71882FG is not set
> > # CONFIG_SENSORS_F75375S is not set
> > # CONFIG_SENSORS_FSCHER is not set
> > # CONFIG_SENSORS_FSCPOS is not set
> > # CONFIG_SENSORS_FSCHMD is not set
> > # CONFIG_SENSORS_GL518SM is not set
> > # CONFIG_SENSORS_GL520SM is not set
> > # CONFIG_SENSORS_CORETEMP is not set
> > # CONFIG_SENSORS_IT87 is not set
> > # CONFIG_SENSORS_LM63 is not set
> > # CONFIG_SENSORS_LM75 is not set
> > # CONFIG_SENSORS_LM77 is not set
> > # CONFIG_SENSORS_LM78 is not set
> > # CONFIG_SENSORS_LM80 is not set
> > # CONFIG_SENSORS_LM83 is not set
> > # CONFIG_SENSORS_LM85 is not set
> > # CONFIG_SENSORS_LM87 is not set
> > # CONFIG_SENSORS_LM90 is not set
> > # CONFIG_SENSORS_LM92 is not set
> > # CONFIG_SENSORS_LM93 is not set
> > # CONFIG_SENSORS_LTC4245 is not set
> > # CONFIG_SENSORS_MAX1619 is not set
> > # CONFIG_SENSORS_MAX6650 is not set
> > # CONFIG_SENSORS_PC87360 is not set
> > # CONFIG_SENSORS_PC87427 is not set
> > # CONFIG_SENSORS_SIS5595 is not set
> > # CONFIG_SENSORS_DME1737 is not set
> > # CONFIG_SENSORS_SMSC47M1 is not set
> > # CONFIG_SENSORS_SMSC47M192 is not set
> > # CONFIG_SENSORS_SMSC47B397 is not set
> > # CONFIG_SENSORS_ADS7828 is not set
> > # CONFIG_SENSORS_THMC50 is not set
> > # CONFIG_SENSORS_VIA686A is not set
> > # CONFIG_SENSORS_VT1211 is not set
> > # CONFIG_SENSORS_VT8231 is not set
> > # CONFIG_SENSORS_W83781D is not set
> > # CONFIG_SENSORS_W83791D is not set
> > # CONFIG_SENSORS_W83792D is not set
> > # CONFIG_SENSORS_W83793 is not set
> > # CONFIG_SENSORS_W83L785TS is not set
> > # CONFIG_SENSORS_W83L786NG is not set
> > # CONFIG_SENSORS_W83627HF is not set
> > # CONFIG_SENSORS_W83627EHF is not set
> > # CONFIG_SENSORS_HDAPS is not set
> > # CONFIG_SENSORS_LIS3LV02D is not set
> > # CONFIG_SENSORS_APPLESMC is not set
> > # CONFIG_HWMON_DEBUG_CHIP is not set
> > CONFIG_THERMAL=y
> > # CONFIG_THERMAL_HWMON is not set
> > # CONFIG_WATCHDOG is not set
> > CONFIG_SSB_POSSIBLE=y
> > 
> > #
> > # Sonics Silicon Backplane
> > #
> > # CONFIG_SSB is not set
> > 
> > #
> > # Multifunction device drivers
> > #
> > # CONFIG_MFD_CORE is not set
> > # CONFIG_MFD_SM501 is not set
> > # CONFIG_HTC_PASIC3 is not set
> > # CONFIG_TWL4030_CORE is not set
> > # CONFIG_MFD_TMIO is not set
> > # CONFIG_PMIC_DA903X is not set
> > # CONFIG_MFD_WM8400 is not set
> > # CONFIG_MFD_WM8350_I2C is not set
> > # CONFIG_MFD_PCF50633 is not set
> > # CONFIG_REGULATOR is not set
> > 
> > #
> > # Multimedia devices
> > #
> > 
> > #
> > # Multimedia core support
> > #
> > # CONFIG_VIDEO_DEV is not set
> > # CONFIG_DVB_CORE is not set
> > # CONFIG_VIDEO_MEDIA is not set
> > 
> > #
> > # Multimedia drivers
> > #
> > CONFIG_DAB=y
> > # CONFIG_USB_DABUSB is not set
> > 
> > #
> > # Graphics support
> > #
> > CONFIG_AGP=y
> > # CONFIG_AGP_ALI is not set
> > # CONFIG_AGP_ATI is not set
> > # CONFIG_AGP_AMD is not set
> > CONFIG_AGP_AMD64=y
> > CONFIG_AGP_INTEL=y
> > # CONFIG_AGP_NVIDIA is not set
> > # CONFIG_AGP_SIS is not set
> > # CONFIG_AGP_SWORKS is not set
> > # CONFIG_AGP_VIA is not set
> > # CONFIG_AGP_EFFICEON is not set
> > CONFIG_DRM=y
> > # CONFIG_DRM_TDFX is not set
> > # CONFIG_DRM_R128 is not set
> > CONFIG_DRM_RADEON=y
> > CONFIG_DRM_I810=y
> > CONFIG_DRM_I830=y
> > # CONFIG_DRM_I915 is not set
> > # CONFIG_DRM_MGA is not set
> > # CONFIG_DRM_SIS is not set
> > # CONFIG_DRM_VIA is not set
> > # CONFIG_DRM_SAVAGE is not set
> > # CONFIG_VGASTATE is not set
> > # CONFIG_VIDEO_OUTPUT_CONTROL is not set
> > # CONFIG_FB is not set
> > # CONFIG_BACKLIGHT_LCD_SUPPORT is not set
> > 
> > #
> > # Display device support
> > #
> > # CONFIG_DISPLAY_SUPPORT is not set
> > 
> > #
> > # Console display driver support
> > #
> > CONFIG_VGA_CONSOLE=y
> > # CONFIG_VGACON_SOFT_SCROLLBACK is not set
> > CONFIG_DUMMY_CONSOLE=y
> > CONFIG_SOUND=y
> > CONFIG_SOUND_OSS_CORE=y
> > CONFIG_SND=y
> > CONFIG_SND_TIMER=y
> > CONFIG_SND_PCM=y
> > CONFIG_SND_RAWMIDI=y
> > CONFIG_SND_SEQUENCER=y
> > CONFIG_SND_SEQ_DUMMY=y
> > CONFIG_SND_OSSEMUL=y
> > CONFIG_SND_MIXER_OSS=y
> > CONFIG_SND_PCM_OSS=y
> > CONFIG_SND_PCM_OSS_PLUGINS=y
> > CONFIG_SND_SEQUENCER_OSS=y
> > CONFIG_SND_RTCTIMER=y
> > CONFIG_SND_SEQ_RTCTIMER_DEFAULT=y
> > # CONFIG_SND_DYNAMIC_MINORS is not set
> > CONFIG_SND_SUPPORT_OLD_API=y
> > CONFIG_SND_VERBOSE_PROCFS=y
> > CONFIG_SND_VERBOSE_PRINTK=y
> > # CONFIG_SND_DEBUG is not set
> > CONFIG_SND_VMASTER=y
> > CONFIG_SND_AC97_CODEC=y
> > CONFIG_SND_DRIVERS=y
> > # CONFIG_SND_DUMMY is not set
> > CONFIG_SND_VIRMIDI=y
> > # CONFIG_SND_MTPAV is not set
> > # CONFIG_SND_MTS64 is not set
> > # CONFIG_SND_SERIAL_U16550 is not set
> > # CONFIG_SND_MPU401 is not set
> > # CONFIG_SND_PORTMAN2X4 is not set
> > # CONFIG_SND_AC97_POWER_SAVE is not set
> > CONFIG_SND_PCI=y
> > # CONFIG_SND_AD1889 is not set
> > # CONFIG_SND_ALS300 is not set
> > # CONFIG_SND_ALS4000 is not set
> > # CONFIG_SND_ALI5451 is not set
> > # CONFIG_SND_ATIIXP is not set
> > # CONFIG_SND_ATIIXP_MODEM is not set
> > # CONFIG_SND_AU8810 is not set
> > # CONFIG_SND_AU8820 is not set
> > # CONFIG_SND_AU8830 is not set
> > # CONFIG_SND_AW2 is not set
> > # CONFIG_SND_AZT3328 is not set
> > # CONFIG_SND_BT87X is not set
> > # CONFIG_SND_CA0106 is not set
> > # CONFIG_SND_CMIPCI is not set
> > # CONFIG_SND_OXYGEN is not set
> > # CONFIG_SND_CS4281 is not set
> > # CONFIG_SND_CS46XX is not set
> > # CONFIG_SND_CS5530 is not set
> > # CONFIG_SND_CS5535AUDIO is not set
> > # CONFIG_SND_DARLA20 is not set
> > # CONFIG_SND_GINA20 is not set
> > # CONFIG_SND_LAYLA20 is not set
> > # CONFIG_SND_DARLA24 is not set
> > # CONFIG_SND_GINA24 is not set
> > # CONFIG_SND_LAYLA24 is not set
> > # CONFIG_SND_MONA is not set
> > # CONFIG_SND_MIA is not set
> > # CONFIG_SND_ECHO3G is not set
> > # CONFIG_SND_INDIGO is not set
> > # CONFIG_SND_INDIGOIO is not set
> > # CONFIG_SND_INDIGODJ is not set
> > # CONFIG_SND_EMU10K1 is not set
> > # CONFIG_SND_EMU10K1X is not set
> > # CONFIG_SND_ENS1370 is not set
> > # CONFIG_SND_ENS1371 is not set
> > # CONFIG_SND_ES1938 is not set
> > # CONFIG_SND_ES1968 is not set
> > # CONFIG_SND_FM801 is not set
> > # CONFIG_SND_HDA_INTEL is not set
> > # CONFIG_SND_HDSP is not set
> > # CONFIG_SND_HDSPM is not set
> > # CONFIG_SND_HIFIER is not set
> > # CONFIG_SND_ICE1712 is not set
> > # CONFIG_SND_ICE1724 is not set
> > CONFIG_SND_INTEL8X0=y
> > # CONFIG_SND_INTEL8X0M is not set
> > # CONFIG_SND_KORG1212 is not set
> > # CONFIG_SND_MAESTRO3 is not set
> > # CONFIG_SND_MIXART is not set
> > # CONFIG_SND_NM256 is not set
> > # CONFIG_SND_PCXHR is not set
> > # CONFIG_SND_RIPTIDE is not set
> > # CONFIG_SND_RME32 is not set
> > # CONFIG_SND_RME96 is not set
> > # CONFIG_SND_RME9652 is not set
> > # CONFIG_SND_SIS7019 is not set
> > # CONFIG_SND_SONICVIBES is not set
> > # CONFIG_SND_TRIDENT is not set
> > # CONFIG_SND_VIA82XX is not set
> > # CONFIG_SND_VIA82XX_MODEM is not set
> > # CONFIG_SND_VIRTUOSO is not set
> > # CONFIG_SND_VX222 is not set
> > # CONFIG_SND_YMFPCI is not set
> > CONFIG_SND_USB=y
> > # CONFIG_SND_USB_AUDIO is not set
> > # CONFIG_SND_USB_USX2Y is not set
> > # CONFIG_SND_USB_CAIAQ is not set
> > # CONFIG_SND_USB_US122L is not set
> > CONFIG_SND_PCMCIA=y
> > # CONFIG_SND_VXPOCKET is not set
> > # CONFIG_SND_PDAUDIOCF is not set
> > # CONFIG_SND_SOC is not set
> > # CONFIG_SOUND_PRIME is not set
> > CONFIG_AC97_BUS=y
> > CONFIG_HID_SUPPORT=y
> > CONFIG_HID=y
> > # CONFIG_HID_DEBUG is not set
> > # CONFIG_HIDRAW is not set
> > 
> > #
> > # USB Input Devices
> > #
> > CONFIG_USB_HID=y
> > # CONFIG_HID_PID is not set
> > # CONFIG_USB_HIDDEV is not set
> > 
> > #
> > # Special HID drivers
> > #
> > CONFIG_HID_COMPAT=y
> > CONFIG_HID_A4TECH=y
> > CONFIG_HID_APPLE=y
> > CONFIG_HID_BELKIN=y
> > CONFIG_HID_CHERRY=y
> > CONFIG_HID_CHICONY=y
> > CONFIG_HID_CYPRESS=y
> > CONFIG_HID_EZKEY=y
> > CONFIG_HID_GYRATION=y
> > CONFIG_HID_LOGITECH=y
> > # CONFIG_LOGITECH_FF is not set
> > # CONFIG_LOGIRUMBLEPAD2_FF is not set
> > CONFIG_HID_MICROSOFT=y
> > CONFIG_HID_MONTEREY=y
> > CONFIG_HID_NTRIG=y
> > CONFIG_HID_PANTHERLORD=y
> > # CONFIG_PANTHERLORD_FF is not set
> > CONFIG_HID_PETALYNX=y
> > CONFIG_HID_SAMSUNG=y
> > CONFIG_HID_SONY=y
> > CONFIG_HID_SUNPLUS=y
> > # CONFIG_GREENASIA_FF is not set
> > CONFIG_HID_TOPSEED=y
> > CONFIG_THRUSTMASTER_FF=m
> > CONFIG_ZEROPLUS_FF=m
> > CONFIG_USB_SUPPORT=y
> > CONFIG_USB_ARCH_HAS_HCD=y
> > CONFIG_USB_ARCH_HAS_OHCI=y
> > CONFIG_USB_ARCH_HAS_EHCI=y
> > CONFIG_USB=y
> > # CONFIG_USB_DEBUG is not set
> > # CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set
> > 
> > #
> > # Miscellaneous USB options
> > #
> > CONFIG_USB_DEVICEFS=y
> > CONFIG_USB_DEVICE_CLASS=y
> > CONFIG_USB_DYNAMIC_MINORS=y
> > # CONFIG_USB_SUSPEND is not set
> > # CONFIG_USB_OTG is not set
> > CONFIG_USB_MON=y
> > # CONFIG_USB_WUSB is not set
> > # CONFIG_USB_WUSB_CBAF is not set
> > 
> > #
> > # USB Host Controller Drivers
> > #
> > # CONFIG_USB_C67X00_HCD is not set
> > CONFIG_USB_EHCI_HCD=y
> > # CONFIG_USB_EHCI_ROOT_HUB_TT is not set
> > # CONFIG_USB_EHCI_TT_NEWSCHED is not set
> > # CONFIG_USB_OXU210HP_HCD is not set
> > # CONFIG_USB_ISP116X_HCD is not set
> > # CONFIG_USB_ISP1760_HCD is not set
> > CONFIG_USB_OHCI_HCD=y
> > # CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
> > # CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
> > CONFIG_USB_OHCI_LITTLE_ENDIAN=y
> > CONFIG_USB_UHCI_HCD=y
> > # CONFIG_USB_SL811_HCD is not set
> > # CONFIG_USB_R8A66597_HCD is not set
> > # CONFIG_USB_WHCI_HCD is not set
> > # CONFIG_USB_HWA_HCD is not set
> > 
> > #
> > # USB Device Class drivers
> > #
> > # CONFIG_USB_ACM is not set
> > CONFIG_USB_PRINTER=y
> > # CONFIG_USB_WDM is not set
> > # CONFIG_USB_TMC is not set
> > 
> > #
> > # NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may also be needed;
> > #
> > 
> > #
> > # see USB_STORAGE Help for more information
> > #
> > CONFIG_USB_STORAGE=y
> > # CONFIG_USB_STORAGE_DEBUG is not set
> > # CONFIG_USB_STORAGE_DATAFAB is not set
> > # CONFIG_USB_STORAGE_FREECOM is not set
> > # CONFIG_USB_STORAGE_ISD200 is not set
> > # CONFIG_USB_STORAGE_USBAT is not set
> > # CONFIG_USB_STORAGE_SDDR09 is not set
> > # CONFIG_USB_STORAGE_SDDR55 is not set
> > # CONFIG_USB_STORAGE_JUMPSHOT is not set
> > # CONFIG_USB_STORAGE_ALAUDA is not set
> > # CONFIG_USB_STORAGE_ONETOUCH is not set
> > # CONFIG_USB_STORAGE_KARMA is not set
> > # CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
> > # CONFIG_USB_LIBUSUAL is not set
> > 
> > #
> > # USB Imaging devices
> > #
> > # CONFIG_USB_MDC800 is not set
> > # CONFIG_USB_MICROTEK is not set
> > 
> > #
> > # USB port drivers
> > #
> > # CONFIG_USB_USS720 is not set
> > # CONFIG_USB_SERIAL is not set
> > 
> > #
> > # USB Miscellaneous drivers
> > #
> > # CONFIG_USB_EMI62 is not set
> > # CONFIG_USB_EMI26 is not set
> > # CONFIG_USB_ADUTUX is not set
> > # CONFIG_USB_SEVSEG is not set
> > # CONFIG_USB_RIO500 is not set
> > # CONFIG_USB_LEGOTOWER is not set
> > # CONFIG_USB_LCD is not set
> > # CONFIG_USB_BERRY_CHARGE is not set
> > # CONFIG_USB_LED is not set
> > # CONFIG_USB_CYPRESS_CY7C63 is not set
> > # CONFIG_USB_CYTHERM is not set
> > # CONFIG_USB_PHIDGET is not set
> > # CONFIG_USB_IDMOUSE is not set
> > # CONFIG_USB_FTDI_ELAN is not set
> > # CONFIG_USB_APPLEDISPLAY is not set
> > # CONFIG_USB_SISUSBVGA is not set
> > # CONFIG_USB_LD is not set
> > # CONFIG_USB_TRANCEVIBRATOR is not set
> > # CONFIG_USB_IOWARRIOR is not set
> > # CONFIG_USB_TEST is not set
> > # CONFIG_USB_ISIGHTFW is not set
> > # CONFIG_USB_VST is not set
> > # CONFIG_USB_GADGET is not set
> > 
> > #
> > # OTG and related infrastructure
> > #
> > # CONFIG_UWB is not set
> > # CONFIG_MMC is not set
> > # CONFIG_MEMSTICK is not set
> > # CONFIG_NEW_LEDS is not set
> > # CONFIG_ACCESSIBILITY is not set
> > CONFIG_INFINIBAND=y
> > # CONFIG_INFINIBAND_USER_MAD is not set
> > # CONFIG_INFINIBAND_USER_ACCESS is not set
> > CONFIG_INFINIBAND_ADDR_TRANS=y
> > # CONFIG_INFINIBAND_MTHCA is not set
> > # CONFIG_INFINIBAND_AMSO1100 is not set
> > # CONFIG_MLX4_INFINIBAND is not set
> > # CONFIG_INFINIBAND_NES is not set
> > # CONFIG_INFINIBAND_IPOIB is not set
> > # CONFIG_INFINIBAND_SRP is not set
> > # CONFIG_INFINIBAND_ISER is not set
> > CONFIG_EDAC=y
> > 
> > #
> > # Reporting subsystems
> > #
> > # CONFIG_EDAC_DEBUG is not set
> > CONFIG_EDAC_MM_EDAC=y
> > # CONFIG_EDAC_AMD76X is not set
> > # CONFIG_EDAC_E7XXX is not set
> > # CONFIG_EDAC_E752X is not set
> > # CONFIG_EDAC_I82875P is not set
> > # CONFIG_EDAC_I82975X is not set
> > # CONFIG_EDAC_I3000 is not set
> > # CONFIG_EDAC_X38 is not set
> > # CONFIG_EDAC_I5400 is not set
> > # CONFIG_EDAC_I82860 is not set
> > # CONFIG_EDAC_R82600 is not set
> > # CONFIG_EDAC_I5000 is not set
> > # CONFIG_EDAC_I5100 is not set
> > # CONFIG_RTC_CLASS is not set
> > # CONFIG_DMADEVICES is not set
> > # CONFIG_AUXDISPLAY is not set
> > # CONFIG_UIO is not set
> > # CONFIG_STAGING is not set
> > CONFIG_X86_PLATFORM_DEVICES=y
> > # CONFIG_ASUS_LAPTOP is not set
> > # CONFIG_TC1100_WMI is not set
> > # CONFIG_SONY_LAPTOP is not set
> > # CONFIG_THINKPAD_ACPI is not set
> > # CONFIG_INTEL_MENLOW is not set
> > # CONFIG_EEEPC_LAPTOP is not set
> > # CONFIG_ACPI_WMI is not set
> > # CONFIG_ACPI_ASUS is not set
> > # CONFIG_ACPI_TOSHIBA is not set
> > 
> > #
> > # Firmware Drivers
> > #
> > # CONFIG_EDD is not set
> > CONFIG_FIRMWARE_MEMMAP=y
> > # CONFIG_DELL_RBU is not set
> > # CONFIG_DCDBAS is not set
> > CONFIG_DMIID=y
> > # CONFIG_ISCSI_IBFT_FIND is not set
> > 
> > #
> > # File systems
> > #
> > CONFIG_EXT2_FS=y
> > CONFIG_EXT2_FS_XATTR=y
> > CONFIG_EXT2_FS_POSIX_ACL=y
> > CONFIG_EXT2_FS_SECURITY=y
> > # CONFIG_EXT2_FS_XIP is not set
> > CONFIG_EXT3_FS=y
> > CONFIG_EXT3_FS_XATTR=y
> > CONFIG_EXT3_FS_POSIX_ACL=y
> > CONFIG_EXT3_FS_SECURITY=y
> > CONFIG_EXT4_FS=y
> > # CONFIG_EXT4DEV_COMPAT is not set
> > CONFIG_EXT4_FS_XATTR=y
> > CONFIG_EXT4_FS_POSIX_ACL=y
> > CONFIG_EXT4_FS_SECURITY=y
> > CONFIG_JBD=y
> > CONFIG_JBD_DEBUG=y
> > CONFIG_JBD2=y
> > CONFIG_JBD2_DEBUG=y
> > CONFIG_FS_MBCACHE=y
> > CONFIG_REISERFS_FS=y
> > # CONFIG_REISERFS_CHECK is not set
> > # CONFIG_REISERFS_PROC_INFO is not set
> > CONFIG_REISERFS_FS_XATTR=y
> > CONFIG_REISERFS_FS_POSIX_ACL=y
> > # CONFIG_REISERFS_FS_SECURITY is not set
> > CONFIG_JFS_FS=y
> > CONFIG_JFS_POSIX_ACL=y
> > # CONFIG_JFS_SECURITY is not set
> > # CONFIG_JFS_DEBUG is not set
> > # CONFIG_JFS_STATISTICS is not set
> > CONFIG_FS_POSIX_ACL=y
> > CONFIG_FILE_LOCKING=y
> > CONFIG_XFS_FS=y
> > # CONFIG_XFS_QUOTA is not set
> > CONFIG_XFS_POSIX_ACL=y
> > # CONFIG_XFS_RT is not set
> > # CONFIG_XFS_DEBUG is not set
> > # CONFIG_GFS2_FS is not set
> > CONFIG_OCFS2_FS=y
> > CONFIG_OCFS2_FS_O2CB=y
> > CONFIG_OCFS2_FS_USERSPACE_CLUSTER=y
> > CONFIG_OCFS2_FS_STATS=y
> > CONFIG_OCFS2_DEBUG_MASKLOG=y
> > # CONFIG_OCFS2_DEBUG_FS is not set
> > # CONFIG_OCFS2_FS_POSIX_ACL is not set
> > # CONFIG_BTRFS_FS is not set
> > CONFIG_DNOTIFY=y
> > CONFIG_INOTIFY=y
> > CONFIG_INOTIFY_USER=y
> > CONFIG_QUOTA=y
> > # CONFIG_QUOTA_NETLINK_INTERFACE is not set
> > CONFIG_PRINT_QUOTA_WARNING=y
> > CONFIG_QUOTA_TREE=y
> > # CONFIG_QFMT_V1 is not set
> > CONFIG_QFMT_V2=y
> > CONFIG_QUOTACTL=y
> > # CONFIG_AUTOFS_FS is not set
> > # CONFIG_AUTOFS4_FS is not set
> > # CONFIG_FUSE_FS is not set
> > 
> > #
> > # CD-ROM/DVD Filesystems
> > #
> > CONFIG_ISO9660_FS=y
> > CONFIG_JOLIET=y
> > CONFIG_ZISOFS=y
> > CONFIG_UDF_FS=y
> > CONFIG_UDF_NLS=y
> > 
> > #
> > # DOS/FAT/NT Filesystems
> > #
> > CONFIG_FAT_FS=y
> > CONFIG_MSDOS_FS=y
> > CONFIG_VFAT_FS=y
> > CONFIG_FAT_DEFAULT_CODEPAGE=437
> > CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
> > # CONFIG_NTFS_FS is not set
> > 
> > #
> > # Pseudo filesystems
> > #
> > CONFIG_PROC_FS=y
> > CONFIG_PROC_KCORE=y
> > CONFIG_PROC_SYSCTL=y
> > CONFIG_PROC_PAGE_MONITOR=y
> > CONFIG_SYSFS=y
> > CONFIG_TMPFS=y
> > # CONFIG_TMPFS_POSIX_ACL is not set
> > # CONFIG_HUGETLBFS is not set
> > # CONFIG_HUGETLB_PAGE is not set
> > CONFIG_CONFIGFS_FS=y
> > CONFIG_MISC_FILESYSTEMS=y
> > # CONFIG_ADFS_FS is not set
> > # CONFIG_AFFS_FS is not set
> > # CONFIG_ECRYPT_FS is not set
> > # CONFIG_HFS_FS is not set
> > # CONFIG_HFSPLUS_FS is not set
> > # CONFIG_BEFS_FS is not set
> > # CONFIG_BFS_FS is not set
> > # CONFIG_EFS_FS is not set
> > # CONFIG_CRAMFS is not set
> > # CONFIG_SQUASHFS is not set
> > # CONFIG_VXFS_FS is not set
> > # CONFIG_MINIX_FS is not set
> > # CONFIG_OMFS_FS is not set
> > # CONFIG_HPFS_FS is not set
> > # CONFIG_QNX4FS_FS is not set
> > # CONFIG_ROMFS_FS is not set
> > # CONFIG_SYSV_FS is not set
> > # CONFIG_UFS_FS is not set
> > CONFIG_NETWORK_FILESYSTEMS=y
> > CONFIG_NFS_FS=y
> > CONFIG_NFS_V3=y
> > CONFIG_NFS_V3_ACL=y
> > CONFIG_NFS_V4=y
> > CONFIG_NFSD=y
> > CONFIG_NFSD_V2_ACL=y
> > CONFIG_NFSD_V3=y
> > CONFIG_NFSD_V3_ACL=y
> > CONFIG_NFSD_V4=y
> > # CONFIG_NFSD_V4_1 is not set
> > CONFIG_LOCKD=y
> > CONFIG_LOCKD_V4=y
> > CONFIG_EXPORTFS=y
> > CONFIG_NFS_ACL_SUPPORT=y
> > CONFIG_NFS_COMMON=y
> > CONFIG_SUNRPC=y
> > CONFIG_SUNRPC_GSS=y
> > CONFIG_SUNRPC_XPRT_RDMA=y
> > # CONFIG_SUNRPC_REGISTER_V4 is not set
> > CONFIG_RPCSEC_GSS_KRB5=y
> > CONFIG_RPCSEC_GSS_SPKM3=y
> > # CONFIG_SMB_FS is not set
> > # CONFIG_CIFS is not set
> > # CONFIG_NCP_FS is not set
> > # CONFIG_CODA_FS is not set
> > # CONFIG_AFS_FS is not set
> > 
> > #
> > # Partition Types
> > #
> > CONFIG_PARTITION_ADVANCED=y
> > # CONFIG_ACORN_PARTITION is not set
> > # CONFIG_OSF_PARTITION is not set
> > # CONFIG_AMIGA_PARTITION is not set
> > # CONFIG_ATARI_PARTITION is not set
> > # CONFIG_MAC_PARTITION is not set
> > CONFIG_MSDOS_PARTITION=y
> > # CONFIG_BSD_DISKLABEL is not set
> > # CONFIG_MINIX_SUBPARTITION is not set
> > # CONFIG_SOLARIS_X86_PARTITION is not set
> > # CONFIG_UNIXWARE_DISKLABEL is not set
> > # CONFIG_LDM_PARTITION is not set
> > # CONFIG_SGI_PARTITION is not set
> > # CONFIG_ULTRIX_PARTITION is not set
> > # CONFIG_SUN_PARTITION is not set
> > # CONFIG_KARMA_PARTITION is not set
> > # CONFIG_EFI_PARTITION is not set
> > # CONFIG_SYSV68_PARTITION is not set
> > CONFIG_NLS=y
> > CONFIG_NLS_DEFAULT="iso8859-1"
> > CONFIG_NLS_CODEPAGE_437=y
> > # CONFIG_NLS_CODEPAGE_737 is not set
> > # CONFIG_NLS_CODEPAGE_775 is not set
> > # CONFIG_NLS_CODEPAGE_850 is not set
> > # CONFIG_NLS_CODEPAGE_852 is not set
> > # CONFIG_NLS_CODEPAGE_855 is not set
> > # CONFIG_NLS_CODEPAGE_857 is not set
> > # CONFIG_NLS_CODEPAGE_860 is not set
> > # CONFIG_NLS_CODEPAGE_861 is not set
> > # CONFIG_NLS_CODEPAGE_862 is not set
> > # CONFIG_NLS_CODEPAGE_863 is not set
> > # CONFIG_NLS_CODEPAGE_864 is not set
> > # CONFIG_NLS_CODEPAGE_865 is not set
> > # CONFIG_NLS_CODEPAGE_866 is not set
> > # CONFIG_NLS_CODEPAGE_869 is not set
> > # CONFIG_NLS_CODEPAGE_936 is not set
> > # CONFIG_NLS_CODEPAGE_950 is not set
> > # CONFIG_NLS_CODEPAGE_932 is not set
> > # CONFIG_NLS_CODEPAGE_949 is not set
> > # CONFIG_NLS_CODEPAGE_874 is not set
> > # CONFIG_NLS_ISO8859_8 is not set
> > # CONFIG_NLS_CODEPAGE_1250 is not set
> > # CONFIG_NLS_CODEPAGE_1251 is not set
> > # CONFIG_NLS_ASCII is not set
> > CONFIG_NLS_ISO8859_1=y
> > # CONFIG_NLS_ISO8859_2 is not set
> > # CONFIG_NLS_ISO8859_3 is not set
> > # CONFIG_NLS_ISO8859_4 is not set
> > # CONFIG_NLS_ISO8859_5 is not set
> > # CONFIG_NLS_ISO8859_6 is not set
> > # CONFIG_NLS_ISO8859_7 is not set
> > # CONFIG_NLS_ISO8859_9 is not set
> > # CONFIG_NLS_ISO8859_13 is not set
> > # CONFIG_NLS_ISO8859_14 is not set
> > # CONFIG_NLS_ISO8859_15 is not set
> > # CONFIG_NLS_KOI8_R is not set
> > # CONFIG_NLS_KOI8_U is not set
> > # CONFIG_NLS_UTF8 is not set
> > CONFIG_DLM=y
> > CONFIG_DLM_DEBUG=y
> > 
> > #
> > # Kernel hacking
> > #
> > CONFIG_TRACE_IRQFLAGS_SUPPORT=y
> > # CONFIG_PRINTK_TIME is not set
> > CONFIG_ENABLE_WARN_DEPRECATED=y
> > CONFIG_ENABLE_MUST_CHECK=y
> > CONFIG_FRAME_WARN=1024
> > CONFIG_MAGIC_SYSRQ=y
> > CONFIG_UNUSED_SYMBOLS=y
> > CONFIG_DEBUG_FS=y
> > # CONFIG_HEADERS_CHECK is not set
> > CONFIG_DEBUG_KERNEL=y
> > # CONFIG_DEBUG_SHIRQ is not set
> > CONFIG_DETECT_SOFTLOCKUP=y
> > # CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
> > CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
> > CONFIG_SCHED_DEBUG=y
> > CONFIG_SCHEDSTATS=y
> > # CONFIG_TIMER_STATS is not set
> > CONFIG_DEBUG_OBJECTS=y
> > # CONFIG_DEBUG_OBJECTS_SELFTEST is not set
> > CONFIG_DEBUG_OBJECTS_FREE=y
> > CONFIG_DEBUG_OBJECTS_TIMERS=y
> > CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT=1
> > CONFIG_DEBUG_SLAB=y
> > CONFIG_DEBUG_SLAB_LEAK=y
> > CONFIG_DEBUG_PREEMPT=y
> > CONFIG_DEBUG_RT_MUTEXES=y
> > CONFIG_DEBUG_PI_LIST=y
> > # CONFIG_RT_MUTEX_TESTER is not set
> > CONFIG_DEBUG_SPINLOCK=y
> > CONFIG_DEBUG_MUTEXES=y
> > CONFIG_DEBUG_LOCK_ALLOC=y
> > CONFIG_PROVE_LOCKING=y
> > CONFIG_LOCKDEP=y
> > # CONFIG_LOCK_STAT is not set
> > # CONFIG_DEBUG_LOCKDEP is not set
> > CONFIG_TRACE_IRQFLAGS=y
> > CONFIG_DEBUG_SPINLOCK_SLEEP=y
> > # CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
> > CONFIG_STACKTRACE=y
> > # CONFIG_DEBUG_KOBJECT is not set
> > # CONFIG_DEBUG_HIGHMEM is not set
> > CONFIG_DEBUG_BUGVERBOSE=y
> > CONFIG_DEBUG_INFO=y
> > CONFIG_DEBUG_VM=y
> > # CONFIG_DEBUG_VIRTUAL is not set
> > CONFIG_DEBUG_WRITECOUNT=y
> > CONFIG_DEBUG_MEMORY_INIT=y
> > CONFIG_DEBUG_LIST=y
> > CONFIG_DEBUG_SG=y
> > # CONFIG_DEBUG_NOTIFIERS is not set
> > CONFIG_ARCH_WANT_FRAME_POINTERS=y
> > CONFIG_FRAME_POINTER=y
> > # CONFIG_BOOT_PRINTK_DELAY is not set
> > # CONFIG_RCU_TORTURE_TEST is not set
> > # CONFIG_RCU_CPU_STALL_DETECTOR is not set
> > # CONFIG_KPROBES_SANITY_TEST is not set
> > # CONFIG_BACKTRACE_SELF_TEST is not set
> > # CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
> > # CONFIG_LKDTM is not set
> > CONFIG_FAULT_INJECTION=y
> > CONFIG_FAILSLAB=y
> > CONFIG_FAIL_PAGE_ALLOC=y
> > CONFIG_FAIL_MAKE_REQUEST=y
> > # CONFIG_FAIL_IO_TIMEOUT is not set
> > CONFIG_FAULT_INJECTION_DEBUG_FS=y
> > # CONFIG_FAULT_INJECTION_STACKTRACE_FILTER is not set
> > CONFIG_LATENCYTOP=y
> > CONFIG_SYSCTL_SYSCALL_CHECK=y
> > CONFIG_USER_STACKTRACE_SUPPORT=y
> > CONFIG_NOP_TRACER=y
> > CONFIG_HAVE_FUNCTION_TRACER=y
> > CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
> > CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
> > CONFIG_HAVE_DYNAMIC_FTRACE=y
> > CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
> > CONFIG_HAVE_HW_BRANCH_TRACER=y
> > CONFIG_TRACER_MAX_TRACE=y
> > CONFIG_RING_BUFFER=y
> > CONFIG_TRACING=y
> > 
> > #
> > # Tracers
> > #
> > CONFIG_FUNCTION_TRACER=y
> > CONFIG_FUNCTION_GRAPH_TRACER=y
> > # CONFIG_IRQSOFF_TRACER is not set
> > # CONFIG_PREEMPT_TRACER is not set
> > CONFIG_SYSPROF_TRACER=y
> > CONFIG_SCHED_TRACER=y
> > CONFIG_CONTEXT_SWITCH_TRACER=y
> > # CONFIG_BOOT_TRACER is not set
> > # CONFIG_TRACE_BRANCH_PROFILING is not set
> > # CONFIG_POWER_TRACER is not set
> > CONFIG_STACK_TRACER=y
> > # CONFIG_HW_BRANCH_TRACER is not set
> > CONFIG_DYNAMIC_FTRACE=y
> > CONFIG_FTRACE_MCOUNT_RECORD=y
> > # CONFIG_FTRACE_STARTUP_TEST is not set
> > # CONFIG_MMIOTRACE is not set
> > # CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
> > CONFIG_DYNAMIC_PRINTK_DEBUG=y
> > # CONFIG_SAMPLES is not set
> > CONFIG_HAVE_ARCH_KGDB=y
> > CONFIG_KGDB=y
> > CONFIG_KGDB_SERIAL_CONSOLE=y
> > # CONFIG_KGDB_TESTS is not set
> > # CONFIG_STRICT_DEVMEM is not set
> > CONFIG_X86_VERBOSE_BOOTUP=y
> > CONFIG_EARLY_PRINTK=y
> > # CONFIG_EARLY_PRINTK_DBGP is not set
> > CONFIG_DEBUG_STACKOVERFLOW=y
> > CONFIG_DEBUG_STACK_USAGE=y
> > # CONFIG_DEBUG_PAGEALLOC is not set
> > # CONFIG_X86_PTDUMP is not set
> > CONFIG_DEBUG_RODATA=y
> > # CONFIG_DEBUG_RODATA_TEST is not set
> > # CONFIG_DEBUG_NX_TEST is not set
> > # CONFIG_4KSTACKS is not set
> > CONFIG_DOUBLEFAULT=y
> > CONFIG_HAVE_MMIOTRACE_SUPPORT=y
> > CONFIG_IO_DELAY_TYPE_0X80=0
> > CONFIG_IO_DELAY_TYPE_0XED=1
> > CONFIG_IO_DELAY_TYPE_UDELAY=2
> > CONFIG_IO_DELAY_TYPE_NONE=3
> > # CONFIG_IO_DELAY_0X80 is not set
> > CONFIG_IO_DELAY_0XED=y
> > # CONFIG_IO_DELAY_UDELAY is not set
> > # CONFIG_IO_DELAY_NONE is not set
> > CONFIG_DEFAULT_IO_DELAY_TYPE=1
> > # CONFIG_DEBUG_BOOT_PARAMS is not set
> > # CONFIG_CPA_DEBUG is not set
> > # CONFIG_OPTIMIZE_INLINING is not set
> > 
> > #
> > # Security options
> > #
> > CONFIG_KEYS=y
> > CONFIG_KEYS_DEBUG_PROC_KEYS=y
> > CONFIG_SECURITY=y
> > # CONFIG_SECURITYFS is not set
> > # CONFIG_SECURITY_NETWORK is not set
> > # CONFIG_SECURITY_PATH is not set
> > # CONFIG_SECURITY_FILE_CAPABILITIES is not set
> > # CONFIG_SECURITY_ROOTPLUG is not set
> > CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR=0
> > CONFIG_CRYPTO=y
> > 
> > #
> > # Crypto core or helper
> > #
> > # CONFIG_CRYPTO_FIPS is not set
> > CONFIG_CRYPTO_ALGAPI=y
> > CONFIG_CRYPTO_ALGAPI2=y
> > CONFIG_CRYPTO_AEAD2=y
> > CONFIG_CRYPTO_BLKCIPHER=y
> > CONFIG_CRYPTO_BLKCIPHER2=y
> > CONFIG_CRYPTO_HASH=y
> > CONFIG_CRYPTO_HASH2=y
> > CONFIG_CRYPTO_RNG2=y
> > CONFIG_CRYPTO_MANAGER=y
> > CONFIG_CRYPTO_MANAGER2=y
> > # CONFIG_CRYPTO_GF128MUL is not set
> > # CONFIG_CRYPTO_NULL is not set
> > # CONFIG_CRYPTO_CRYPTD is not set
> > # CONFIG_CRYPTO_AUTHENC is not set
> > # CONFIG_CRYPTO_TEST is not set
> > 
> > #
> > # Authenticated Encryption with Associated Data
> > #
> > # CONFIG_CRYPTO_CCM is not set
> > # CONFIG_CRYPTO_GCM is not set
> > # CONFIG_CRYPTO_SEQIV is not set
> > 
> > #
> > # Block modes
> > #
> > CONFIG_CRYPTO_CBC=y
> > # CONFIG_CRYPTO_CTR is not set
> > # CONFIG_CRYPTO_CTS is not set
> > CONFIG_CRYPTO_ECB=y
> > # CONFIG_CRYPTO_LRW is not set
> > # CONFIG_CRYPTO_PCBC is not set
> > # CONFIG_CRYPTO_XTS is not set
> > 
> > #
> > # Hash modes
> > #
> > CONFIG_CRYPTO_HMAC=y
> > # CONFIG_CRYPTO_XCBC is not set
> > 
> > #
> > # Digest
> > #
> > CONFIG_CRYPTO_CRC32C=y
> > # CONFIG_CRYPTO_CRC32C_INTEL is not set
> > # CONFIG_CRYPTO_MD4 is not set
> > CONFIG_CRYPTO_MD5=y
> > CONFIG_CRYPTO_MICHAEL_MIC=y
> > # CONFIG_CRYPTO_RMD128 is not set
> > # CONFIG_CRYPTO_RMD160 is not set
> > # CONFIG_CRYPTO_RMD256 is not set
> > # CONFIG_CRYPTO_RMD320 is not set
> > CONFIG_CRYPTO_SHA1=y
> > # CONFIG_CRYPTO_SHA256 is not set
> > # CONFIG_CRYPTO_SHA512 is not set
> > # CONFIG_CRYPTO_TGR192 is not set
> > # CONFIG_CRYPTO_WP512 is not set
> > 
> > #
> > # Ciphers
> > #
> > CONFIG_CRYPTO_AES=y
> > # CONFIG_CRYPTO_AES_586 is not set
> > # CONFIG_CRYPTO_ANUBIS is not set
> > CONFIG_CRYPTO_ARC4=y
> > # CONFIG_CRYPTO_BLOWFISH is not set
> > # CONFIG_CRYPTO_CAMELLIA is not set
> > CONFIG_CRYPTO_CAST5=y
> > # CONFIG_CRYPTO_CAST6 is not set
> > CONFIG_CRYPTO_DES=y
> > # CONFIG_CRYPTO_FCRYPT is not set
> > # CONFIG_CRYPTO_KHAZAD is not set
> > # CONFIG_CRYPTO_SALSA20 is not set
> > # CONFIG_CRYPTO_SALSA20_586 is not set
> > # CONFIG_CRYPTO_SEED is not set
> > # CONFIG_CRYPTO_SERPENT is not set
> > # CONFIG_CRYPTO_TEA is not set
> > # CONFIG_CRYPTO_TWOFISH is not set
> > # CONFIG_CRYPTO_TWOFISH_586 is not set
> > 
> > #
> > # Compression
> > #
> > # CONFIG_CRYPTO_DEFLATE is not set
> > # CONFIG_CRYPTO_LZO is not set
> > 
> > #
> > # Random Number Generation
> > #
> > # CONFIG_CRYPTO_ANSI_CPRNG is not set
> > CONFIG_CRYPTO_HW=y
> > # CONFIG_CRYPTO_DEV_PADLOCK is not set
> > # CONFIG_CRYPTO_DEV_GEODE is not set
> > # CONFIG_CRYPTO_DEV_HIFN_795X is not set
> > CONFIG_HAVE_KVM=y
> > CONFIG_VIRTUALIZATION=y
> > # CONFIG_KVM is not set
> > # CONFIG_LGUEST is not set
> > CONFIG_VIRTIO=y
> > CONFIG_VIRTIO_RING=y
> > CONFIG_VIRTIO_PCI=y
> > # CONFIG_VIRTIO_BALLOON is not set
> > 
> > #
> > # Library routines
> > #
> > CONFIG_BITREVERSE=y
> > CONFIG_GENERIC_FIND_FIRST_BIT=y
> > CONFIG_GENERIC_FIND_NEXT_BIT=y
> > CONFIG_GENERIC_FIND_LAST_BIT=y
> > # CONFIG_CRC_CCITT is not set
> > CONFIG_CRC16=y
> > CONFIG_CRC_T10DIF=y
> > CONFIG_CRC_ITU_T=y
> > CONFIG_CRC32=y
> > # CONFIG_CRC7 is not set
> > CONFIG_LIBCRC32C=y
> > CONFIG_ZLIB_INFLATE=y
> > CONFIG_PLIST=y
> > CONFIG_HAS_IOMEM=y
> > CONFIG_HAS_IOPORT=y
> > CONFIG_HAS_DMA=y

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Benny Halevy]

On Mar. 31, 2009, 0:15 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Mon, Mar 30, 2009 at 10:38:47PM +0300, Benny Halevy wrote:
>> On Mar. 30, 2009, 22:07 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>> On Mon, Mar 30, 2009 at 10:03:39PM +0300, Benny Halevy wrote:
>>>> On Mar. 30, 2009, 21:33 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>> On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
>>>>>> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>>>>> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
>>>>>>>> Hi Bruce,
>>>>>>>>
>>>>>>>> Here's the latest server patches implementing the NFSv4.1
>>>>>>>> Sessions features.
>>>>>>>>
>>>>>>>> This patchset is based over your for-2.6.30 branch
>>>>>>>> and is also available from
>>>>>>>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
>>>>>>> There's a few merge conflicts with my current for-2.6.30--probably my
>>>>>>> fault for not pushing that out recently enough, apologies.  Would you
>>>>>>> mind updating?
>>>>>>>
>>>>>>> --b.
>>>>>>>
>>>>>> Sure. Here's a rebased version with two minor changes (see below)
>>>>> On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a new NULL
>>>>> dereference in the callback code.  Looks like it probably happened while
>>>>> running connectathon over NFSv4.0 with krb5p.  That's all I've figured out so
>>>> Weird. nfs4_xdr_dec_cb_recall+0x4e doesn't seem like a valid IP
>>>> for nfsd.ko @a564667.  Bruce, can you please send me you .config file?
>>> I'm building without CONFIG_NFSD_V4_1 for now; .config appended.
>> It looks like rpc_res would be NULL without CONFIG_NFSD_V4_1.
>> Does the following patch help?
> 
> Yep!

Cool.  I'll add this as a SQUASHME patch.

Benny

> 
> --b.
> 
>> Benny
>>
>> git diff --stat -p
>>  fs/nfsd/nfs4callback.c |    8 +++++---
>>  1 files changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
>> index 02e0a61..10c6a80 100644
>> --- a/fs/nfsd/nfs4callback.c
>> +++ b/fs/nfsd/nfs4callback.c
>> @@ -441,9 +441,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
>>  	status = decode_cb_compound_hdr(&xdr, &hdr);
>>  	if (status)
>>  		goto out;
>> -	status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
>> -	if (status)
>> -		goto out;
>> +	if (rpc_res && rpc_res->res_seq) {
>> +		status = decode_cb_sequence(&xdr, rpc_res->res_seq, rqstp);
>> +		if (status)
>> +			goto out;
>> +	}
>>  	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
>>  out:
>>  	return status;
>>

RE: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Labiaga, Ricardo]

> -----Original Message-----
> From: Benny Halevy [mailto:bhalevy@panasas.com]
> Sent: Monday, March 30, 2009 12:04 PM
> To: J. Bruce Fields; Labiaga, Ricardo
> Cc: NFS list; pNFS Mailing List
> Subject: Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30
> 
> 
> On Mar. 30, 2009, 21:33 +0300, "J. Bruce Fields"
<bfields@fieldses.org>
> wrote:
> > On Sat, Mar 28, 2009 at 11:28:50AM +0300, Benny Halevy wrote:
> >> On Mar. 28, 2009, 3:01 +0300, "J. Bruce Fields"
<bfields@fieldses.org>
> wrote:
> >>> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> >>>> Hi Bruce,
> >>>>
> >>>> Here's the latest server patches implementing the NFSv4.1
> >>>> Sessions features.
> >>>>
> >>>> This patchset is based over your for-2.6.30 branch
> >>>> and is also available from
> >>>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> >>> There's a few merge conflicts with my current for-2.6.30--probably
my
> >>> fault for not pushing that out recently enough, apologies.  Would
you
> >>> mind updating?
> >>>
> >>> --b.
> >>>
> >> Sure. Here's a rebased version with two minor changes (see below)
> >
> > On the latest version of nfsd41-for-2.6.30 (a564667..) I'm getting a
new
> NULL
> > dereference in the callback code.  Looks like it probably happened
while
> > running connectathon over NFSv4.0 with krb5p.  That's all I've
figured
> out so
> 
> Weird. nfs4_xdr_dec_cb_recall+0x4e doesn't seem like a valid IP
> for nfsd.ko @a564667.  Bruce, can you please send me you .config file?

It is indeed a valid instruction for the NFS server callback.  Recall
that the NFS server uses the client side RPC to send and process
replies.  nfsd4_cb_recall() initializes
nfs4_cb_procedures[CB_RECALL].p_decode to nfs4_xdr_dec_cb_recall().
This is later called by rpcauth_unwrap_resp() which is called by
call_decode() when the reply to the callback arrives.

> 
> We know the current implementation is still incorrect for krb5,
> but I don't think it's supposed to crash either.
> Ricardo, have you run into this by any chance?

I have not run into this, but I have not run this with v4.1 not compiled
in either.  I believe you nailed the problem in the diff you provided in
a later email.

- ricardo

> 
> Benny
> 
> > far.
> >
> > --b.
> >
> > BUG: unable to handle kernel NULL pointer dereference at (null)
> > IP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200
> > *pde = 00000000
> > Oops: 0000 [#1] PREEMPT
> > last sysfs file: /sys/kernel/uevent_seqnum
> > Modules linked in:
> >
> > Pid: 3949, comm: nfs4_cb_recall Not tainted
(2.6.29-rc8-00312-ga564667
> #39)
> > EIP: 0060:[<c03e4e1e>] EFLAGS: 00010286 CPU: 0
> > EIP is at nfs4_xdr_dec_cb_recall+0x4e/0x200
> > EAX: c7bb22a0 EBX: c7bb2298 ECX: c7bb22a0 EDX: c7bb22a4
> > ESI: 00000000 EDI: c66e3000 EBP: c5281ea4 ESP: c5281e6c
> >  DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
> > Process nfs4_cb_recall (pid: 3949, ti=c5280000 task=c66a6b60
> task.ti=c5280000)
> > Stack:
> >  c66a6b60 00000000 c5281eb4 c086cab4 c0aa0380 00000001 c11bd4a4
c7bb22a4
> >  c66e302c c7bb22ac c66e302c 00000000 c6cf14e8 c66e3000 c5281ecc
c08199c3
> >  c0819aae 00000000 c5281ee8 c5281ef0 c03e4dd0 c6cf14e8 c7bb2298
c03e4dd0
> > Call Trace:
> >  [<c086cab4>] ? schedule+0x354/0x540
> >  [<c08199c3>] ? rpcauth_unwrap_resp+0x63/0x90
> >  [<c0819aae>] ? rpcauth_checkverf+0x2e/0x70
> >  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
> >  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
> >  [<c08122ce>] ? call_decode+0x1ae/0x820
> >  [<c086d125>] ? out_of_line_wait_on_bit+0x65/0x80
> >  [<c0818720>] ? rpc_wait_bit_killable+0x0/0x40
> >  [<c03e4dd0>] ? nfs4_xdr_dec_cb_recall+0x0/0x200
> >  [<c0818e62>] ? __rpc_execute+0x92/0x290
> >  [<c086f4fc>] ? _spin_unlock+0x2c/0x50
> >  [<c08186c7>] ? rpc_set_active+0x67/0x80
> >  [<c081907e>] ? rpc_execute+0x1e/0x30
> >  [<c08129e5>] ? rpc_run_task+0x35/0x70
> >  [<c0812b40>] ? rpc_call_sync+0x40/0x70
> >  [<c03e51e0>] ? nfsd4_cb_recall+0x70/0x130
> >  [<c086cab4>] ? schedule+0x354/0x540
> >  [<c0247d1b>] ? trace_hardirqs_on+0xb/0x10
> >  [<c03dfd40>] ? do_recall+0x0/0x20
> >  [<c03dfd54>] ? do_recall+0x14/0x20
> >  [<c023941f>] ? kthread+0x3f/0x70
> >  [<c02393e0>] ? kthread+0x0/0x70
> >  [<c0203b67>] ? kernel_thread_helper+0x7/0x10
> > Code: 43 00 ba 08 00 00 00 8d 45 e4 e8 0e c1 43 00 85 c0 74 3a 8b 50
04
> 8d 45 e4 0f ca 83 c2 04 e8 fa c0 43 00 85 c0 0f 84 8a 00 00 00 <8b> 06
8b
> 00 85 c0 75 32 ba 04 00 00 00 8d 45 e4 e8 cd fe ff ff
> > EIP: [<c03e4e1e>] nfs4_xdr_dec_cb_recall+0x4e/0x200 SS:ESP
0068:c5281e6c
> > ---[ end trace 2724475d9856cb6c ]---
> >

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[J. Bruce Fields]

On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
> Hi Bruce,
> 
> Here's the latest server patches implementing the NFSv4.1
> Sessions features.
> 
> This patchset is based over your for-2.6.30 branch
> and is also available from
> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30

I ran the v4.0 pynfs on that branch and got some new failures:

ACC3     st_access.testNoFh                                       : FAILURE
           value=10064 not in enum nfsstat4
CLOSE7   st_close.testNoCfh                                       : FAILURE
           value=10064 not in enum nfsstat4
CMT3     st_commit.testNoFh                                       : FAILURE
           value=10064 not in enum nfsstat4
CR8      st_create.testNoFh                                       : FAILURE
           value=10064 not in enum nfsstat4

Looks like NFS4ERR_SEQUENCE_POS is leaking out to nfsv4 code for some
reason?

(Some similar results omitted.)

Also there's a bunch of stuff like this:

NVF1a    st_nverify.testMandLink                                  : FAILURE
           NVerifying mandatory attributes against getattr should
           return NFS4ERR_SAME, instead got NFS4_OK
NVF1d    st_nverify.testMandDir                                   : FAILURE
           NVerifying mandatory attributes against getattr should
           return NFS4ERR_SAME, instead got NFS4_OK

Probably the expansion of the attribute bitmask is exposing some
preexisting bug in the 4.0 code, but let's try to figure out how to fix
that.

I haven't investigated either of these any closer yet.

--b.

Re: [PATCH 0/47] NFSv4.1 Sessions server code for 2.6.30

[Benny Halevy]

On Mar. 28, 2009, 4:17 +0300, "J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Fri, Mar 27, 2009 at 05:58:45AM +0300, Benny Halevy wrote:
>> Hi Bruce,
>>
>> Here's the latest server patches implementing the NFSv4.1
>> Sessions features.
>>
>> This patchset is based over your for-2.6.30 branch
>> and is also available from
>> git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.30
> 
> I ran the v4.0 pynfs on that branch and got some new failures:
> 
> ACC3     st_access.testNoFh                                       : FAILURE
>            value=10064 not in enum nfsstat4
> CLOSE7   st_close.testNoCfh                                       : FAILURE
>            value=10064 not in enum nfsstat4
> CMT3     st_commit.testNoFh                                       : FAILURE
>            value=10064 not in enum nfsstat4
> CR8      st_create.testNoFh                                       : FAILURE
>            value=10064 not in enum nfsstat4
> 
> Looks like NFS4ERR_SEQUENCE_POS is leaking out to nfsv4 code for some
> reason?

Right, we checked ALLOWED_AS_FIRST_OP for minorversion 0 too.

> 
> (Some similar results omitted.)
> 
> Also there's a bunch of stuff like this:
> 
> NVF1a    st_nverify.testMandLink                                  : FAILURE
>            NVerifying mandatory attributes against getattr should
>            return NFS4ERR_SAME, instead got NFS4_OK
> NVF1d    st_nverify.testMandDir                                   : FAILURE
>            NVerifying mandatory attributes against getattr should
>            return NFS4ERR_SAME, instead got NFS4_OK
> 
> Probably the expansion of the attribute bitmask is exposing some
> preexisting bug in the 4.0 code, but let's try to figure out how to fix
> that.

A change to nfsd4_encode_fattr to encode bitmaps of variable length
(rather than 2) broke _nfsd4_verify.

> 
> I haven't investigated either of these any closer yet.
> 
> --b.

The following two patches fix these regressions:
[PATCH 1/2] SQUASHME: nfsd41: do not verify nfserr_sequence_pos for minorversion 0
[PATCH 2/2] nfsd: dynamically skip encoded fattr bitmap in _nfsd4_verify

Benny

[PATCH 1/2] SQUASHME: nfsd41: do not verify nfserr_sequence_pos for minorversion 0

[Benny Halevy]

squash into "nfsd41: enforce NFS4ERR_SEQUENCE_POS operation order rules"

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c |   17 ++++++++++++++---
 1 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 6d7d7c5..20583ab 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -894,6 +894,18 @@ nfsd4_enc_no_page_replay(struct nfsd4_compoundargs *args,
 	return op->status;
 }
 
+static bool nfs41_op_ordering_ok(struct nfsd4_compoundargs *args)
+{
+#if defined(CONFIG_NFSD_V4_1)
+	if (args->minorversion && args->opcnt > 0) {
+		struct nfsd4_op *op = &args->ops[0];
+		return (op->status == nfserr_op_illegal) ||
+		       (nfsd4_ops[op->opnum].op_flags & ALLOWED_AS_FIRST_OP);
+	}
+#endif /* CONFIG_NFSD_V4_1 */
+	return true;
+}
+
 /*
  * COMPOUND call.
  */
@@ -933,9 +945,8 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,
 	if (args->minorversion > NFSD_SUPPORTED_MINOR_VERSION)
 		goto out;
 
-	op = &args->ops[0];
-	if (args->opcnt > 0 && op->status != nfserr_op_illegal &&
-	    !(nfsd4_ops[op->opnum].op_flags & ALLOWED_AS_FIRST_OP)) {
+	if (!nfs41_op_ordering_ok(args)) {
+		op = &args->ops[0];
 		op->status = nfserr_sequence_pos;
 		goto encode_op;
 	}
-- 
1.6.2.1

[PATCH 2/2] nfsd: dynamically skip encoded fattr bitmap in _nfsd4_verify

[Benny Halevy]

_nfsd4_verify currently skips 3 words from the encoded buffer begining.
With support for 3-word attr bitmaps in nfsd41, nfsd4_encode_fattr
may encode 1, 2, or 3 words, and not always 2 as it used to be, hence
we need to find out where to skip using the encoded bitmap length.

Note: This patch may be applied over pre-nfsd41 nfsd.
move this patch in front of "nfsd41: support for 3-word long attribute bitmask"

Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/nfsd/nfs4proc.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 20583ab..a393d38 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -801,7 +801,8 @@ _nfsd4_verify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (status)
 		goto out_kfree;
 
-	p = buf + 3;
+	/* skip bitmap */
+	p = buf + 1 + ntohl(buf[0]);
 	status = nfserr_not_same;
 	if (ntohl(*p++) != verify->ve_attrlen)
 		goto out_kfree;
-- 
1.6.2.1