All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH pynfs 00/17] prep for flex file layout server
@ 2014-06-04 21:01 Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session Weston Andros Adamson
                   ` (17 more replies)
  0 siblings, 18 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

These patches are in preparation for the pynfs implementation of a flex
file layout server.

I already have patches for a working flex file server, but I figure those
should wait until the spec is complete ;)

I pushed these patches to my linux-nfs.org tree on branch 'ff_prep':

  git://git.linux-nfs.org/projects/dros/pynfs.git

The prep includes:

 - patches that enable pynfs mds using real v4.1 DSes (on the backend,
   not usable from clients as there is no stateid sharing between mds and ds)

 - fix several bugs in pnfs path

 - move .x files to subdir 'xdrdef' - the nfs4.1 directory was getting crowded
   and we're going to add several more .x files along with all of the
   autogenerated files.  Note this change touches server tests too.

 - clean up the nfs4server's output - no longer print out EVERY part of EVERY
   operation which was very efficient at hiding tracebacks and made pynfs
   even slower. Old functionality is now switched on with -v flag. Also
   added an in-between mode "-s" that prints a summary of whats going on.

 - support for NFSv3 backend communication - this is not actually used yet,
   but can be tested with file layout by always returning
   NFS4ERR_LAYOUTUNAVAILABLE in layoutget and using DataServer3 class instead
   of DataServer41

Weston Andros Adamson (17):
  4.1 client: reclaim_complete after create_session
  4.1 server: service RECLAIM_COMPLETE operations
  dataserver: only catch connection error
  4.1 server: avoid traceback in DS disconnect()
  move .x files to subdir 'xdrdef'
  4.1 client: remove unused imports
  4.1 server: add -v flag & silence random output
  4.1 server: add -s option to print summary of ops
  dataserver: make generic interface to ops
  dataserver: don't import * from nfs4 specific mods
  4.1 server: move nfs4_ops.py to nfs_ops.py
  add mntv3, portmapv2 and nfsv3 .x files
  dataserver: separate generic and 4.1 code
  4.1 server: add support for NFSv3 data servers
  4.1 server: get rid of old op_getdeviceinfo
  rpc: on socket error, close and mark pipe inactive
  nfs3clnt: reconnect when sending on inactive pipe

 nfs4.1/block.py                             |    8 +-
 nfs4.1/client41tests/environment.py         |    4 +-
 nfs4.1/config.py                            |    8 +-
 nfs4.1/dataserver.py                        |  306 ++-
 nfs4.1/fs.py                                |   38 +-
 nfs4.1/nfs3client.py                        |  185 ++
 nfs4.1/nfs4.x                               | 3290 ---------------------------
 nfs4.1/nfs4.x.diff                          |   48 -
 nfs4.1/nfs4_ops.py                          |   61 -
 nfs4.1/nfs4client.py                        |   48 +-
 nfs4.1/nfs4commoncode.py                    |   10 +-
 nfs4.1/nfs4lib.py                           |   62 +-
 nfs4.1/nfs4server.py                        |  166 +-
 nfs4.1/nfs4state.py                         |   15 +-
 nfs4.1/nfs_ops.py                           |   89 +
 nfs4.1/pnfs_block.x                         |  149 --
 nfs4.1/sctrl.x                              |  145 --
 nfs4.1/server41tests/environment.py         |    4 +-
 nfs4.1/server41tests/st_block.py            |    4 +-
 nfs4.1/server41tests/st_compound.py         |    4 +-
 nfs4.1/server41tests/st_create_session.py   |    4 +-
 nfs4.1/server41tests/st_current_stateid.py  |    8 +-
 nfs4.1/server41tests/st_debug.py            |    4 +-
 nfs4.1/server41tests/st_delegation.py       |    4 +-
 nfs4.1/server41tests/st_destroy_clientid.py |    2 +-
 nfs4.1/server41tests/st_destroy_session.py  |    4 +-
 nfs4.1/server41tests/st_exchange_id.py      |    4 +-
 nfs4.1/server41tests/st_getdevicelist.py    |    4 +-
 nfs4.1/server41tests/st_lookup.py           |    2 +-
 nfs4.1/server41tests/st_lookupp.py          |    2 +-
 nfs4.1/server41tests/st_open.py             |    8 +-
 nfs4.1/server41tests/st_putfh.py            |    2 +-
 nfs4.1/server41tests/st_reboot.py           |    4 +-
 nfs4.1/server41tests/st_reclaim_complete.py |    2 +-
 nfs4.1/server41tests/st_rename.py           |    4 +-
 nfs4.1/server41tests/st_secinfo.py          |    4 +-
 nfs4.1/server41tests/st_secinfo_no_name.py  |    4 +-
 nfs4.1/server41tests/st_sequence.py         |    4 +-
 nfs4.1/server41tests/st_trunking.py         |    4 +-
 nfs4.1/server41tests/st_verify.py           |    2 +-
 nfs4.1/server_exports.py                    |    7 +-
 nfs4.1/setup.py                             |   33 +-
 nfs4.1/testclient.py                        |    2 +-
 nfs4.1/testserver.py                        |    2 +-
 nfs4.1/xdrdef/__init__.py                   |    0
 nfs4.1/xdrdef/mnt3.x                        |   68 +
 nfs4.1/xdrdef/nfs3.x                        |  818 +++++++
 nfs4.1/xdrdef/nfs4.x                        | 3290 +++++++++++++++++++++++++++
 nfs4.1/xdrdef/nfs4.x.diff                   |   48 +
 nfs4.1/xdrdef/pnfs_block.x                  |  149 ++
 nfs4.1/xdrdef/portmap.x                     |   51 +
 nfs4.1/xdrdef/sctrl.x                       |  145 ++
 rpc/rpc.py                                  |   26 +-
 53 files changed, 5337 insertions(+), 4022 deletions(-)
 create mode 100644 nfs4.1/nfs3client.py
 delete mode 100644 nfs4.1/nfs4.x
 delete mode 100644 nfs4.1/nfs4.x.diff
 delete mode 100644 nfs4.1/nfs4_ops.py
 create mode 100644 nfs4.1/nfs_ops.py
 delete mode 100644 nfs4.1/pnfs_block.x
 delete mode 100644 nfs4.1/sctrl.x
 create mode 100644 nfs4.1/xdrdef/__init__.py
 create mode 100644 nfs4.1/xdrdef/mnt3.x
 create mode 100644 nfs4.1/xdrdef/nfs3.x
 create mode 100644 nfs4.1/xdrdef/nfs4.x
 create mode 100644 nfs4.1/xdrdef/nfs4.x.diff
 create mode 100644 nfs4.1/xdrdef/pnfs_block.x
 create mode 100644 nfs4.1/xdrdef/portmap.x
 create mode 100644 nfs4.1/xdrdef/sctrl.x

-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply	[flat|nested] 31+ messages in thread

* [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-05  2:26   ` J. Bruce Fields
  2014-06-04 21:01 ` [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations Weston Andros Adamson
                   ` (16 subsequent siblings)
  17 siblings, 1 reply; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Send RECLAIM_COMPLETE after CREATE_SESSION. This enables backend
communication to v4.1 linux nfsd servers (i/o through MDS only).

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/nfs4client.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
index e750728..0d94a42 100644
--- a/nfs4.1/nfs4client.py
+++ b/nfs4.1/nfs4client.py
@@ -367,6 +367,7 @@ class ClientRecord(object):
         self.seqid = inc_u32(csr.csr_sequence) # XXX Do we need to check this?
         sess = SessionRecord(csr, self)
         self.c.sessions[sess.sessionid] = sess
+        sess.compound([op.reclaim_complete(FALSE)])
         return sess
 
     def _cb_hook(self, prefix, opname, funct):
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-05  2:29   ` J. Bruce Fields
  2014-06-04 21:01 ` [PATCH pynfs 03/17] dataserver: only catch connection error Weston Andros Adamson
                   ` (15 subsequent siblings)
  17 siblings, 1 reply; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Just return ok!

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/nfs4server.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
index 65fb9af..3607dc0 100755
--- a/nfs4.1/nfs4server.py
+++ b/nfs4.1/nfs4server.py
@@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
         with find_state(env, arg.deleg_stateid, allow_0=False) as state:
             state.delegreturn()
         return encode_status(NFS4_OK)
+
+    def op_reclaim_complete(self, arg, env):
+        return encode_status(NFS4_OK)
     
     def op_getdevicelist(self, arg, env): # STUB
         check_session(env)
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 03/17] dataserver: only catch connection error
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 04/17] 4.1 server: avoid traceback in DS disconnect() Weston Andros Adamson
                   ` (14 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

This specifically checks for the error being reported, otherwise a
other exceptions like a programming error  will be caught and treated
as connection errors.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index 65e4f55..b8dd903 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -9,6 +9,7 @@ import nfs4client
 import hashlib
 import sys
 import nfs4_ops as op
+import socket
 
 log = logging.getLogger("Dataserver Manager")
 
@@ -197,7 +198,7 @@ class DSDevice(object):
                     ds = DataServer(server, port, path, mdsds=self.mdsds,
                                     multipath_servers=server_list)
                     self.list.append(ds)
-                except:
+                except socket.error:
                     log.critical("cannot access %s:%i/%s" %
                                  (server, port, '/'.join(path)))
                     sys.exit(1)
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 04/17] 4.1 server: avoid traceback in DS disconnect()
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (2 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 03/17] dataserver: only catch connection error Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 06/17] 4.1 client: remove unused imports Weston Andros Adamson
                   ` (13 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Session objects have no destroy method.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index b8dd903..175dd95 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -58,7 +58,7 @@ class DataServer(object):
         self.make_root()
 
     def disconnect(self):
-        self.sess.destroy()
+        pass
 
     def execute(self, ops, exceptions=[], delay=5, maxretries=3):
         """ execute the NFS call
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 06/17] 4.1 client: remove unused imports
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (3 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 04/17] 4.1 server: avoid traceback in DS disconnect() Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 07/17] 4.1 server: add -v flag & silence random output Weston Andros Adamson
                   ` (12 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/nfs4client.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
index 91c698d..0904577 100644
--- a/nfs4.1/nfs4client.py
+++ b/nfs4.1/nfs4client.py
@@ -8,15 +8,13 @@ from xdrdef.sctrl_pack import SCTRLPacker, SCTRLUnpacker
 import nfs4_ops as op
 import time, struct
 import threading
-import collections
 import hmac
-from locking import Lock
 from nfs4commoncode import CBCompoundState as CompoundState, \
      cb_encode_status as encode_status, \
      cb_encode_status_by_name as encode_status_by_name
 
 
-import sys, traceback
+import traceback
 import logging
 logging.basicConfig(level=logging.INFO,
                     format="%(levelname)-7s:%(name)s:%(message)s")
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 07/17] 4.1 server: add -v flag & silence random output
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (4 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 06/17] 4.1 client: remove unused imports Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 08/17] 4.1 server: add -s option to print summary of ops Weston Andros Adamson
                   ` (11 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

By default the nfs4server spews so much information to stdout/stderr
that it's almost useless.  Add the -v flag (aka --verbose) to enable
the old output.

Also remove some random prints and change some to log.info().

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py |  3 +--
 nfs4.1/fs.py         |  4 ----
 nfs4.1/nfs4client.py | 11 ++--------
 nfs4.1/nfs4server.py | 61 +++++++++++++++++++++++-----------------------------
 nfs4.1/nfs4state.py  |  1 -
 5 files changed, 30 insertions(+), 50 deletions(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index 7f5f72c..d697631 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -181,7 +181,7 @@ class DSDevice(object):
                 line = line.strip()
                 if not line or line.startswith('#'):
                     continue
-                print "Analyzing: %r" % line
+                log.info("Analyzing: %r" % line)
                 try:
                     server_list, path = nfs4lib.parse_nfs_url(line)
                 except:
@@ -191,7 +191,6 @@ class DSDevice(object):
                 # for now, just use the last path for local connections
                 server, port = server_list[-1]
                 server_list = server_list[:-1]
-                print server, port, path
                 try:
                     log.info("Adding dataserver ip:%s port:%s path:%s" %
                              (server, port, '/'.join(path)))
diff --git a/nfs4.1/fs.py b/nfs4.1/fs.py
index 3d63ce5..6ef283b 100644
--- a/nfs4.1/fs.py
+++ b/nfs4.1/fs.py
@@ -13,7 +13,6 @@ from xdrdef.nfs4_pack import NFS4Packer
 log_o = logging.getLogger("fs.obj")
 log_fs = logging.getLogger("fs")
 logging.addLevelName(5, "FUNCT")
-log_fs.setLevel(20)
 
 class MetaData(object):
     """Contains everything that needs to be stored
@@ -743,7 +742,6 @@ class ConfigObj(FSObject):
             line = line.strip()
             if line and not line.startswith("#"):
                 lines.append(line)
-        print lines
         if len(lines) != 1:
             self._reset()
             return
@@ -1154,7 +1152,6 @@ class LayoutFSObj(FSObject):
                                             disk_offset,
                                             e.state))
         block_layout = pnfs_block_layout4(elist)
-        print block_layout
         p = block.Packer()
         p.pack_pnfs_block_layout4(block_layout)
 ##         if self.id <= 4:
@@ -1194,7 +1191,6 @@ class LayoutFSObj(FSObject):
                 log_o.exception("Problem decoding opaque")
                 raise NFS4Error(NFS4ERR_BADLAYOUT, tag="Error decoding opaque")
             upd_list = update.blu_commit_list
-            print upd_list
         # Error check
         for e in upd_list:
             if e.bex_state != block.PNFS_BLOCK_READWRITE_DATA:
diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
index 0904577..e504362 100644
--- a/nfs4.1/nfs4client.py
+++ b/nfs4.1/nfs4client.py
@@ -19,9 +19,7 @@ import logging
 logging.basicConfig(level=logging.INFO,
                     format="%(levelname)-7s:%(name)s:%(message)s")
 log_cb = logging.getLogger("nfs.client.cb")
-log_cb.setLevel(logging.DEBUG)
 
-SHOW_TRAFFIC = True # Debugging aid, prints out client traffic
 class NFS4Client(rpc.Client, rpc.Server):
     def __init__(self, host='localhost', port=2049, minorversion=1, ctrl_proc=16):
         rpc.Client.__init__(self, 100003, 4)
@@ -76,9 +74,7 @@ class NFS4Client(rpc.Client, rpc.Server):
             pipe = self.c1
         p = packer(check_enum=checks, check_array=checks)
         c4 = COMPOUND4args(tag, version, ops)
-        if SHOW_TRAFFIC:
-            print
-            print c4
+        log_cb.info("compound args = %r" % (c4,))
         p.pack_COMPOUND4args(c4)
         return self.send_call(pipe, 1, p.get_buffer(), credinfo)
 
@@ -86,19 +82,16 @@ class NFS4Client(rpc.Client, rpc.Server):
         xid = self.compound_async(*args, **kwargs)
         pipe = kwargs.get("pipe", None)
         res = self.listen(xid, pipe=pipe)
-        if SHOW_TRAFFIC:
-            print res
+        log_cb.info("compound result = %r" % (res,))
         return res
     
     def listen(self, xid, pipe=None, timeout=10.0):
         if pipe is None:
             pipe = self.c1
         header, data = pipe.listen(xid, timeout)
-        #print "HEADER", header
         if data:
             p = nfs4lib.FancyNFS4Unpacker(data)
             data = p.unpack_COMPOUND4res()
-        #print "DATA", repr(data)
         return data
 
     def handle_0(self, data, cred):
diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
index 75276f9..67adbf1 100755
--- a/nfs4.1/nfs4server.py
+++ b/nfs4.1/nfs4server.py
@@ -21,14 +21,11 @@ from nfs4commoncode import CompoundState, encode_status, encode_status_by_name
 from fs import RootFS, ConfigFS
 from config import ServerConfig, ServerPerClientConfig, OpsConfigServer, Actions
 
-logging.basicConfig(level=logging.INFO,
+logging.basicConfig(level=logging.WARN,
                     format="%(levelname)-7s:%(name)s:%(message)s")
 log_41 = logging.getLogger("nfs.server")
-log_41.setLevel(logging.DEBUG)
-log_41.setLevel(9)
 
 log_cfg = logging.getLogger("nfs.server.opconfig")
-log_cfg.setLevel(20)
 
 ##################################################
 # Set various global constants and magic numbers #
@@ -116,9 +113,6 @@ class Recording(object):
     def add(self, call, reply):
         """Add call and reply strings to records"""
         if self.on:
-            print "Adding"
-            print repr(call)
-            print repr(reply)
             self.queue.appendleft((call, reply))
 
     def set_stamp(self, stamp):
@@ -269,13 +263,13 @@ class VerboseDict(dict):
         self.config = config
 
     def __setitem__(self, key, value):
-        if 0 or self.config.debug_state:
-            print "+++ Adding client.state[%r]" % key
+        if self.config.debug_state:
+            log_41.info("+++ Adding client.state[%r]" % key)
         dict.__setitem__(self, key, value)
 
     def __delitem__(self, key):
-        if 0 or self.config.debug_state:
-            print "+++ Removing client.state[%r]" % key
+        if self.config.debug_state:
+            log_41.info("+++ Removing client.state[%r]" % key)
         dict.__delitem__(self, key)
         
 class ClientRecord(object):
@@ -526,6 +520,13 @@ class NFS4Server(rpc.Server):
         port = kwargs.pop("port", NFS4_PORT)
         self.is_mds = kwargs.pop("is_mds", False)
         self.is_ds = kwargs.pop("is_ds", False)
+
+        self.verbose = kwargs.pop('verbose', False)
+        if self.verbose:
+            log_41.setLevel(logging.DEBUG) # XXX redundant?
+            log_41.setLevel(9)
+            log_cfg.setLevel(20)
+
         rpc.Server.__init__(self, prog=NFS4_PROGRAM, versions=[4], port=port,
                             **kwargs)
         self.root = RootFS().root # Root of exported filesystem tree
@@ -552,7 +553,7 @@ class NFS4Server(rpc.Server):
             rpc.Server.start(self)
         except KeyboardInterrupt:
             # Put user into console where can look at state of server
-            if not self.config.catch_ctrlc:
+            if not self.config.catch_ctrlc or not self.verbose:
                 raise
             import code
             import readline
@@ -1330,9 +1331,9 @@ class NFS4Server(rpc.Server):
         sid, deleg, flags = self.open_file(existing, arg.owner,
                                     arg.share_access, arg.share_deny)
         env.set_cfh(existing, sid)
-        if 0 or env.session.client.config.debug_state:
-            print "+++ client(id=%i).state =" %  env.session.client.clientid
-            print env.session.client.state
+        if env.session.client.config.debug_state:
+            log_41.info("+++ client(id=%i).state = %r" %
+                        (env.session.client.clientid, env.session.client.state))
         res = OPEN4resok(sid, cinfo, flags, bitmask, deleg)
         return encode_status(NFS4_OK, res)
 
@@ -1452,14 +1453,13 @@ class NFS4Server(rpc.Server):
         ret_dict = {}
         info = nfs4lib.attr_info
         for attr in attrs:
-            print "handling fattr4_%s : " % nfs4lib.attr_name(attr),
             if attr not in info:
                 # Ignore unknown attributes
-                print "Unknown"
+                log_41.info("Skipping unknown attr: %s" % (attr,))
                 continue
             if not info[attr].readable:
                 # XXX How deal with write-only attrs?
-                print "Write only"
+                log_41.info("Skipping write only attr: %s" % (attr,))
                 continue
             # Attributes hide in different places, call the place 'base'
             if info[attr].from_fs:
@@ -1471,24 +1471,14 @@ class NFS4Server(rpc.Server):
             name = "fattr4_%s" % nfs4lib.attr_name(attr)
             if hasattr(base, name) and (obj.fs.fattr4_supported_attrs & 1<<attr): # STUB we should be able to remove hasattr
                 ret_dict[attr] = getattr(base, name)
-                print ret_dict[attr]
             else:
                 if ignore:
                     # Must ignore for GETATTR (and READDIR) per 15.1
-                    print "ignored"
-                    if name == "fattr4_mounted_on_fileid":
-                        print base == obj
-                        print base.fattr4_mounted_on_fileid
-                    if name == "fattr4_layout_blksize":
-                        print base == obj
-                        print hasattr(base, name)
-                        print obj.fs.fattr4_supported_attrs
-                        print 1<<attr
-                        print obj.fs.fattr4_supported_attrs & 1<<attr
+                    log_41.info("ignored attr %s" % (name,))
                     continue
                 else:
                     # This is for VERIFY/NVERIFY
-                    print "NOT SUPP"
+                    log_41.info("attr NOT SUPP %s" % (name,))
                     raise NFS4Error(NFS4ERR_ATTRNOTSUPP)
         obj.fattr4_rdattr_error = NFS4_OK # XXX STUB Handle correctly
         return ret_dict
@@ -1956,14 +1946,14 @@ class NFS4Server(rpc.Server):
                 calls.append(call)
             if arg.dir & xdrdef.sctrl_const.DIR_REPLY:
                 replies.append(reply)
-        print calls
-        print replies
+        #print calls
+        #print replies
         grabres = xdrdef.sctrl_type.GRABres(calls, replies)
         return xdrdef.sctrl_const.CTRLSTAT_OK, \
                xdrdef.sctrl_type.resdata_t(arg.ctrlop, grab = grabres)
 
     def ctrl_illegal(self, arg):
-        print "ILLEGAL"
+        #print "ILLEGAL"
         return xdrdef.sctrl_const.CTRLSTAT_ILLEGAL, xdrdef.sctrl_type.resdata_t(arg.ctrlop)
         
     def op_setclientid(self, arg, env):
@@ -2070,6 +2060,8 @@ def scan_options():
                     )
     p.add_option("-r", "--reset", action="store_true", default=False,
                  help="Reset and clear any disk-based filesystems")
+    p.add_option("-v", "--verbose", action="store_true", default=False,
+                 help="Print debug info to screen and enter interpreter on ^C")
     p.add_option("--use_block", action="store_true", default=False,
                  help="Mount a block-pnfs fs")
     p.add_option("--use_files", action="store_true", default=False,
@@ -2102,7 +2094,8 @@ if __name__ == "__main__":
         locking.DEBUG = True
     S = NFS4Server(port=opts.port,
                    is_mds=opts.use_block or opts.use_files,
-                   is_ds = opts.is_ds)
+                   is_ds = opts.is_ds,
+                   verbose = opts.verbose)
     read_exports(S, opts)
     if True:
         S.start()
diff --git a/nfs4.1/nfs4state.py b/nfs4.1/nfs4state.py
index 1294083..2f3cd59 100644
--- a/nfs4.1/nfs4state.py
+++ b/nfs4.1/nfs4state.py
@@ -13,7 +13,6 @@ import rpc
 import logging
 
 log = logging.getLogger("nfs.server.state")
-log.setLevel(10)
 
 POSIXLOCK = False
 
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 08/17] 4.1 server: add -s option to print summary of ops
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (5 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 07/17] 4.1 server: add -v flag & silence random output Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 09/17] dataserver: make generic interface to ops Weston Andros Adamson
                   ` (10 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

As a middle ground between verbose (-v) mode and silent mode, add
"-s" (aka "--show_summary") to print one line per operation.

The summary mode will print a "role" header whenever the role changes.

Example output - mds starts, talks to v3 ds (.200), client (.11) mounts,
touches an existing file:

call v3 172.16.200.200:2049
  access
Mounting (2, 6) on '/files'

handle v4.1 ::ffff:172.16.200.11:758
  exchange_id
  create_session
  sequence, reclaim_complete
  sequence, putrootfh, secinfo_no_name
  sequence, putrootfh, getfh, getattr
  sequence, putfh, getattr
  (repeated 6 times)
  sequence, putfh, access, getattr
  sequence, putfh, lookup, getfh, getattr
  (repeated 1 times)
  sequence, putfh, secinfo -> NFS4ERR_NOTSUPP
  sequence, putfh, getattr
  (repeated 4 times)
  sequence, putfh, access, getattr

call v3 172.16.200.200:2049
  create -> NFS3ERR_EXIST
  lookup
  getattr
  (repeated 1 times)

handle v4.1 ::ffff:172.16.200.11:758
  sequence, putfh, open, getfh, access, getattr

call v3 172.16.200.200:2049
  getattr
  (repeated 1 times)

handle v4.1 ::ffff:172.16.200.11:758
  sequence, putfh, setattr, getattr
  sequence, putfh, close, getattr

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py     | 16 ++++++++++------
 nfs4.1/nfs4client.py     |  7 ++++++-
 nfs4.1/nfs4server.py     | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 nfs4.1/server_exports.py |  7 ++++---
 4 files changed, 66 insertions(+), 11 deletions(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index d697631..9b0462d 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -13,8 +13,8 @@ import socket
 
 log = logging.getLogger("Dataserver Manager")
 
-class DataServer(object):
-    def __init__(self, server, port, path, flavor=rpc.AUTH_SYS, active=True, mdsds=True, multipath_servers=None):
+class DataServer41(object):
+    def __init__(self, server, port, path, flavor=rpc.AUTH_SYS, active=True, mdsds=True, multipath_servers=None, summary=None):
         self.mdsds = mdsds
         self.server = server
         self.port = int(port)
@@ -32,6 +32,8 @@ class DataServer(object):
         else:
             self.multipath_servers = []
 
+        self.summary = summary
+
         if active:
             self.up()
 
@@ -47,7 +49,8 @@ class DataServer(object):
         # only support root with AUTH_SYS for now
         s1 = rpc.security.instance(rpc.AUTH_SYS)
         self.cred1 = s1.init_cred(uid=0, gid=0)
-        self.c1 = nfs4client.NFS4Client(self.server, self.port)
+        self.c1 = nfs4client.NFS4Client(self.server, self.port,
+                                        summary=self.summary)
         self.c1.set_cred(self.cred1)
         self.c1.null()
         c = self.c1.new_client("DS.init_%s" % self.server)
@@ -172,7 +175,7 @@ class DSDevice(object):
         self.address_body = None # set by load()
         self.mdsds = mdsds # if you are both the DS and the MDS we are the only server
 
-    def load(self, filename):
+    def load(self, filename, server_obj):
         """ Read dataservers from configuration file:
         where each line has format e.g. server[:[port][/path]]
         """
@@ -194,8 +197,9 @@ class DSDevice(object):
                 try:
                     log.info("Adding dataserver ip:%s port:%s path:%s" %
                              (server, port, '/'.join(path)))
-                    ds = DataServer(server, port, path, mdsds=self.mdsds,
-                                    multipath_servers=server_list)
+                    ds = DataServer41(server, port, path, mdsds=self.mdsds,
+                                    multipath_servers=server_list,
+                                    summary=server_obj.summary)
                     self.list.append(ds)
                 except socket.error:
                     log.critical("cannot access %s:%i/%s" %
diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
index e504362..f5d2006 100644
--- a/nfs4.1/nfs4client.py
+++ b/nfs4.1/nfs4client.py
@@ -21,7 +21,7 @@ logging.basicConfig(level=logging.INFO,
 log_cb = logging.getLogger("nfs.client.cb")
 
 class NFS4Client(rpc.Client, rpc.Server):
-    def __init__(self, host='localhost', port=2049, minorversion=1, ctrl_proc=16):
+    def __init__(self, host='localhost', port=2049, minorversion=1, ctrl_proc=16, summary=None):
         rpc.Client.__init__(self, 100003, 4)
         self.prog = 0x40000000
         self.versions = [1] # List of supported versions of prog
@@ -36,6 +36,7 @@ class NFS4Client(rpc.Client, rpc.Server):
         self.c1 = self.connect(self.server_address)
         self.sessions = {} # XXX Really, this should be per server
         self.ctrl_proc = ctrl_proc
+        self.summary = summary
 
     def set_cred(self, credinfo):
         self.default_cred = credinfo
@@ -83,6 +84,10 @@ class NFS4Client(rpc.Client, rpc.Server):
         pipe = kwargs.get("pipe", None)
         res = self.listen(xid, pipe=pipe)
         log_cb.info("compound result = %r" % (res,))
+        if self.summary:
+            self.summary.show_op('call v4.1 %s:%s' % self.server_address,
+                [ nfs_opnum4[a.argop].lower()[3:] for a in args[0] ],
+                nfsstat4[res.status])
         return res
     
     def listen(self, xid, pipe=None, timeout=10.0):
diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
index 67adbf1..088e8c4 100755
--- a/nfs4.1/nfs4server.py
+++ b/nfs4.1/nfs4server.py
@@ -502,6 +502,41 @@ class Slot(object):
 
     # STUB - for client, need to track slot usage
 
+class SummaryOutput:
+    def __init__(self, enabled=True):
+        self._enabled = enabled
+        self._last = None
+        self._last_role = None
+        self._repeat_count = 0
+
+    def show_op(self, role, opnames, status):
+        if not self._enabled:
+            return
+
+        summary_line = "  %s" % ', '.join(opnames)
+
+        if status != "NFS4_OK" and status != "NFS3_OK":
+            summary_line += " -> %s" % (status,)
+
+        print_summary_line = True
+        if summary_line != self._last or role != self._last_role:
+            if self._last and self._repeat_count:
+                print "  (repeated %u times)" % self._repeat_count
+            self._last = summary_line
+            self._repeat_count = 0
+        else:
+            print_summary_line = False
+            self._repeat_count += 1
+
+        if self._last_role != role:
+            print
+            print role
+            self._last_role = role
+
+        if print_summary_line:
+            print summary_line
+
+
 ##################################################
 # The primary class - it is excessively long     #
 ##################################################
@@ -527,6 +562,8 @@ class NFS4Server(rpc.Server):
             log_41.setLevel(9)
             log_cfg.setLevel(20)
 
+        self.summary = SummaryOutput(kwargs.pop('show_summary', False))
+
         rpc.Server.__init__(self, prog=NFS4_PROGRAM, versions=[4], port=port,
                             **kwargs)
         self.root = RootFS().root # Root of exported filesystem tree
@@ -776,6 +813,7 @@ class NFS4Server(rpc.Server):
             return env
         # Handle the individual operations
         status = NFS4_OK
+        opnames = []
         for arg in args.argarray:
             opname = nfs_opnum4.get(arg.argop, 'op_illegal')
             log_41.info("*** %s (%d) ***" % (opname, arg.argop))
@@ -805,10 +843,14 @@ class NFS4Server(rpc.Server):
                     result = encode_status_by_name(opname.lower()[3:],
                                                    NFS4ERR_SERVERFAULT)
             env.results.append(result)
+            opnames.append(opname.lower()[3:])
             status = result.status
             if status != NFS4_OK:
                 break
         log_41.info("Replying.  Status %s (%d)" % (nfsstat4[status], status))
+        client_addr = '%s:%s' % cred.connection._s.getpeername()[:2]
+        self.summary.show_op('handle v4.1 %s' % client_addr,
+                             opnames, nfsstat4[status])
         return env
 
     def delete_session(self, session, sessionid):
@@ -2062,6 +2104,8 @@ def scan_options():
                  help="Reset and clear any disk-based filesystems")
     p.add_option("-v", "--verbose", action="store_true", default=False,
                  help="Print debug info to screen and enter interpreter on ^C")
+    p.add_option("-s", "--show_summary", action="store_true", default=False,
+                 help="Print short summary of operations")
     p.add_option("--use_block", action="store_true", default=False,
                  help="Mount a block-pnfs fs")
     p.add_option("--use_files", action="store_true", default=False,
@@ -2095,7 +2139,8 @@ if __name__ == "__main__":
     S = NFS4Server(port=opts.port,
                    is_mds=opts.use_block or opts.use_files,
                    is_ds = opts.is_ds,
-                   verbose = opts.verbose)
+                   verbose = opts.verbose,
+                   show_summary = opts.show_summary)
     read_exports(S, opts)
     if True:
         S.start()
diff --git a/nfs4.1/server_exports.py b/nfs4.1/server_exports.py
index d96b27b..ef857ee 100644
--- a/nfs4.1/server_exports.py
+++ b/nfs4.1/server_exports.py
@@ -15,7 +15,7 @@ def mount_stuff(server, opts):
         E = BlockLayoutFS(5, backing_device=dev)
         server.mount(E, path="/block")
     if opts.use_files:
-        dservers = _load_dataservers(opts.dataservers, server.is_ds and server.is_mds)
+        dservers = _load_dataservers(opts.dataservers, server)
         if dservers is None:
             return
         F = FileLayoutFS(6, dservers)
@@ -33,7 +33,8 @@ def _create_simple_block_dev():
     c1 = Concat([s3, s1])
     return BlockVolume(c1)
 
-def _load_dataservers(file, connect_to_ds=False):
+def _load_dataservers(filename, server):
+    connect_to_ds = server.is_ds and server.is_mds
     dss = DSDevice(connect_to_ds)
-    dss.load(file)
+    dss.load(filename, server)
     return dss;
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 09/17] dataserver: make generic interface to ops
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (6 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 08/17] 4.1 server: add -s option to print summary of ops Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 10/17] dataserver: don't import * from nfs4 specific mods Weston Andros Adamson
                   ` (9 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Hide the execute method to make DataServer objects more generic.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 41 ++++++++++++++++++++++++++++++++++-------
 nfs4.1/fs.py         | 24 ++++--------------------
 2 files changed, 38 insertions(+), 27 deletions(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index 9b0462d..5a3b851 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -63,7 +63,7 @@ class DataServer41(object):
     def disconnect(self):
         pass
 
-    def execute(self, ops, exceptions=[], delay=5, maxretries=3):
+    def _execute(self, ops, exceptions=[], delay=5, maxretries=3):
         """ execute the NFS call
         If an error code is specified in the exceptions it means that the
         caller wants to handle the error himself
@@ -117,16 +117,16 @@ class DataServer41(object):
         kind = createtype4(NF4DIR)
         for comp in self.path:
             existing_path.append(comp)
-            res = self.execute(nfs4lib.use_obj(existing_path),
+            res = self._execute(nfs4lib.use_obj(existing_path),
                                exceptions=[NFS4ERR_NOENT])
             if res.status == NFS4ERR_NOENT:
                 cr_ops = nfs4lib.use_obj(existing_path[:-1]) + \
                     [op.create(kind, comp, attrs)]
-                self.execute(cr_ops)
-        res = self.execute(nfs4lib.use_obj(self.path) + [op.getfh()])
+                self._execute(cr_ops)
+        res = self._execute(nfs4lib.use_obj(self.path) + [op.getfh()])
         self.path_fh = res.resarray[-1].object
         need = ACCESS4_READ | ACCESS4_LOOKUP | ACCESS4_MODIFY | ACCESS4_EXTEND
-        res = self.execute(nfs4lib.use_obj(self.path_fh) + [op.access(need)])
+        res = self._execute(nfs4lib.use_obj(self.path_fh) + [op.access(need)])
         if res.resarray[-1].access != need:
             raise RuntimeError
         # XXX clean DS directory
@@ -146,7 +146,7 @@ class DataServer41(object):
             open_op = op.open(seqid, access, deny,
                               open_owner4(self.sess.client.clientid, owner),
                               openflag, open_claim4(CLAIM_NULL, name))
-            res = self.execute(nfs4lib.use_obj(self.path_fh) + [open_op, op.getfh()], exceptions=[NFS4ERR_EXIST])
+            res = self._execute(nfs4lib.use_obj(self.path_fh) + [open_op, op.getfh()], exceptions=[NFS4ERR_EXIST])
             if res.status == NFS4_OK:
                  ds_fh = res.resarray[-1].opgetfh.resok4.object
                  ds_openstateid = stateid4(0, res.resarray[-2].stateid.other)
@@ -162,10 +162,37 @@ class DataServer41(object):
         seqid=0 #FIXME: seqid must be !=0
         fh, stateid = self.filehandles[mds_fh]
         ops = [op.putfh(fh)] + [op.close(seqid, stateid)]
-        res = self.execute(ops)
+        res = self._execute(ops)
         # ignoring return
         del self.filehandles[mds_fh]
 
+    def read(self, fh, pos, count):
+        ops = [op.putfh(fh),
+               op.read(nfs4lib.state00, pos, count)]
+        # There are all sorts of error handling issues here
+        res = self._execute(ops)
+        data = res.resarray[-1].data
+        return data
+
+    def write(self, fh, pos, data):
+        ops = [op.putfh(fh),
+               op.write(nfs4lib.state00, pos, FILE_SYNC4, data)]
+        # There are all sorts of error handling issues here
+        res = self._execute(ops)
+
+    def truncate(self, fh, size):
+        ops = [op.putfh(fh),
+               op.setattr(nfs4lib.state00, {FATTR4_SIZE: size})]
+        res = self._execute(ops)
+
+    def get_size(self, fh):
+        ops = [op.putfh(fh),
+               op.getattr(1L << FATTR4_SIZE)]
+        res = self._execute(ops)
+        attrdict = res.resarray[-1].obj_attributes
+        return attrdict.get(FATTR4_SIZE, 0)
+
+
 class DSDevice(object):
     def __init__(self, mdsds):
         self.list = [] # list of DataServer instances
diff --git a/nfs4.1/fs.py b/nfs4.1/fs.py
index 6ef283b..8fc49ef 100644
--- a/nfs4.1/fs.py
+++ b/nfs4.1/fs.py
@@ -1567,12 +1567,7 @@ class FilelayoutVolWrapper(object):
         self._pos = 0
 
     def read(self, count):
-        # STUB stateid0 is illegal to a ds
-        ops = [op.putfh(self._fh),
-               op.read(nfs4lib.state00, self._pos, count)]
-        # There are all sorts of error handling issues here
-        res = self._ds.execute(ops)
-        data = res.resarray[-1].data
+        data = self._ds.read(self._fh, self._pos, count)
         self._pos += len(data)
         return data
 
@@ -1580,25 +1575,14 @@ class FilelayoutVolWrapper(object):
         self._pos = offset
 
     def write(self, data):
-        ops = [op.putfh(self._fh),
-               op.write(nfs4lib.state00, self._pos, FILE_SYNC4, data)]
-        # There are all sorts of error handling issues here
-        res = self._ds.execute(ops)
+        self._ds.write(self._fh, self._pos, data)
         self._pos += len(data)
-        return
 
     def truncate(self, size):
-        ops = [op.putfh(self._fh),
-               op.setattr(nfs4lib.state00, {FATTR4_SIZE: size})]
-        res = self._ds.execute(ops)
-        return
+        self._ds.truncate(self._fh, size)
 
     def get_size(self):
-        ops = [op.putfh(self._fh),
-               op.getattr(1L << FATTR4_SIZE)]
-        res = self._ds.execute(ops)
-        attrdict = res.resarray[-1].obj_attributes
-        return attrdict.get(FATTR4_SIZE, 0)
+        return self._ds.get_size(self._fh)
 
 ################################################
 
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 10/17] dataserver: don't import * from nfs4 specific mods
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (7 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 09/17] dataserver: make generic interface to ops Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:01 ` [PATCH pynfs 11/17] 4.1 server: move nfs4_ops.py to nfs_ops.py Weston Andros Adamson
                   ` (8 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

don't pollute the global namespace with constants and types from nfs4_*

this is in preparation for NFSv3 DSes.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 60 ++++++++++++++++++++++++++--------------------------
 1 file changed, 30 insertions(+), 30 deletions(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index 5a3b851..a825615 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -1,8 +1,8 @@
 import rpc
 import nfs4lib
-from xdrdef.nfs4_type import *
+import xdrdef.nfs4_type as type4
 from xdrdef.nfs4_pack import NFS4Packer
-from xdrdef.nfs4_const import *
+import xdrdef.nfs4_const as const4
 import time
 import logging
 import nfs4client
@@ -56,7 +56,7 @@ class DataServer41(object):
         c = self.c1.new_client("DS.init_%s" % self.server)
         # This is a hack to ensure MDS/DS communication path is at least
         # as wide as the client/MDS channel (at least for linux client)
-        fore_attrs = channel_attrs4(0, 16384, 16384, 2868, 8, 8, [])
+        fore_attrs = type4.channel_attrs4(0, 16384, 16384, 2868, 8, 8, [])
         self.sess = c.create_session(fore_attrs=fore_attrs)
         self.make_root()
 
@@ -68,12 +68,12 @@ class DataServer41(object):
         If an error code is specified in the exceptions it means that the
         caller wants to handle the error himself
         """
-        retry_errors = [NFS4ERR_DELAY, NFS4ERR_GRACE]
-        state_errors = [NFS4ERR_STALE_CLIENTID, NFS4ERR_BADSESSION,
-                        NFS4ERR_BADSLOT, NFS4ERR_DEADSESSION]
+        retry_errors = [const4.NFS4ERR_DELAY, const4.NFS4ERR_GRACE]
+        state_errors = [const4.NFS4ERR_STALE_CLIENTID, const4.NFS4ERR_BADSESSION,
+                        const4.NFS4ERR_BADSLOT, const4.NFS4ERR_DEADSESSION]
         while True:
             res = self.sess.compound(ops)
-            if res.status == NFS4_OK or res.status in exceptions:
+            if res.status == const4.NFS4_OK or res.status in exceptions:
                 return res
             elif res.status in retry_errors:
                 if maxretries > 0:
@@ -95,7 +95,7 @@ class DataServer41(object):
         uaddr = '.'.join([self.server,
                           str(self.port >> 8),
                           str(self.port & 0xff)])
-        return netaddr4(self.proto, uaddr)
+        return type4.netaddr4(self.proto, uaddr)
 
     def get_multipath_netaddr4s(self):
         netaddr4s = []
@@ -108,24 +108,24 @@ class DataServer41(object):
             if server.find(':') >= 0:
                 proto = "tcp6"
 
-            netaddr4s.append(netaddr4(proto, uaddr))
+            netaddr4s.append(type4.netaddr4(proto, uaddr))
         return netaddr4s
 
 
-    def make_root(self, attrs={FATTR4_MODE:0777}):
+    def make_root(self, attrs={const4.FATTR4_MODE:0777}):
         existing_path = []
-        kind = createtype4(NF4DIR)
+        kind = type4.createtype4(const4.NF4DIR)
         for comp in self.path:
             existing_path.append(comp)
             res = self._execute(nfs4lib.use_obj(existing_path),
-                               exceptions=[NFS4ERR_NOENT])
-            if res.status == NFS4ERR_NOENT:
+                               exceptions=[const4.NFS4ERR_NOENT])
+            if res.status == const4.NFS4ERR_NOENT:
                 cr_ops = nfs4lib.use_obj(existing_path[:-1]) + \
                     [op.create(kind, comp, attrs)]
                 self._execute(cr_ops)
         res = self._execute(nfs4lib.use_obj(self.path) + [op.getfh()])
         self.path_fh = res.resarray[-1].object
-        need = ACCESS4_READ | ACCESS4_LOOKUP | ACCESS4_MODIFY | ACCESS4_EXTEND
+        need = const4.ACCESS4_READ | const4.ACCESS4_LOOKUP | const4.ACCESS4_MODIFY | const4.ACCESS4_EXTEND
         res = self._execute(nfs4lib.use_obj(self.path_fh) + [op.access(need)])
         if res.resarray[-1].access != need:
             raise RuntimeError
@@ -135,25 +135,25 @@ class DataServer41(object):
         return hashlib.sha1("%r" % mds_fh).hexdigest()
 
     def open_file(self, mds_fh, seqid=0,
-                  access=OPEN4_SHARE_ACCESS_BOTH, deny=OPEN4_SHARE_DENY_NONE,
-                  attrs={FATTR4_MODE: 0777}, owner = "mds", mode=GUARDED4):
+                  access=const4.OPEN4_SHARE_ACCESS_BOTH, deny=const4.OPEN4_SHARE_DENY_NONE,
+                  attrs={const4.FATTR4_MODE: 0777}, owner = "mds", mode=const4.GUARDED4):
         verifier = self.sess.c.verifier
-        openflag = openflag4(OPEN4_CREATE, createhow4(mode, attrs, verifier))
+        openflag = type4.openflag4(const4.OPEN4_CREATE, type4.createhow4(mode, attrs, verifier))
         name = self.fh_to_name(mds_fh)
         while True:
             if mds_fh in self.filehandles:
                 return
             open_op = op.open(seqid, access, deny,
-                              open_owner4(self.sess.client.clientid, owner),
-                              openflag, open_claim4(CLAIM_NULL, name))
-            res = self._execute(nfs4lib.use_obj(self.path_fh) + [open_op, op.getfh()], exceptions=[NFS4ERR_EXIST])
-            if res.status == NFS4_OK:
+                              type4.open_owner4(self.sess.client.clientid, owner),
+                              openflag, type4.open_claim4(const4.CLAIM_NULL, name))
+            res = self._execute(nfs4lib.use_obj(self.path_fh) + [open_op, op.getfh()], exceptions=[const4.NFS4ERR_EXIST])
+            if res.status == const4.NFS4_OK:
                  ds_fh = res.resarray[-1].opgetfh.resok4.object
-                 ds_openstateid = stateid4(0, res.resarray[-2].stateid.other)
+                 ds_openstateid = type4.stateid4(0, res.resarray[-2].stateid.other)
                  self.filehandles[mds_fh] = (ds_fh, ds_openstateid)
                  return
-            elif res.status == NFS4ERR_EXIST:
-                 openflag = openflag4(OPEN4_NOCREATE)
+            elif res.status == const4.NFS4ERR_EXIST:
+                 openflag = type4.openflag4(const4.OPEN4_NOCREATE)
             else:
                 raise RuntimeError
 
@@ -176,26 +176,26 @@ class DataServer41(object):
 
     def write(self, fh, pos, data):
         ops = [op.putfh(fh),
-               op.write(nfs4lib.state00, pos, FILE_SYNC4, data)]
+               op.write(nfs4lib.state00, pos, const4.FILE_SYNC4, data)]
         # There are all sorts of error handling issues here
         res = self._execute(ops)
 
     def truncate(self, fh, size):
         ops = [op.putfh(fh),
-               op.setattr(nfs4lib.state00, {FATTR4_SIZE: size})]
+               op.setattr(nfs4lib.state00, {const4.FATTR4_SIZE: size})]
         res = self._execute(ops)
 
     def get_size(self, fh):
         ops = [op.putfh(fh),
-               op.getattr(1L << FATTR4_SIZE)]
+               op.getattr(1L << const4.FATTR4_SIZE)]
         res = self._execute(ops)
         attrdict = res.resarray[-1].obj_attributes
-        return attrdict.get(FATTR4_SIZE, 0)
+        return attrdict.get(const4.FATTR4_SIZE, 0)
 
 
 class DSDevice(object):
     def __init__(self, mdsds):
-        self.list = [] # list of DataServer instances
+        self.list = [] # list of DataServer41 instances
         # STUB only one data group supported for now
         self.devid = 0
         self.active = 0
@@ -247,7 +247,7 @@ class DSDevice(object):
                 netaddrs.append(multipath)
                 stripe_indices.append(index)
                 index = index + 1
-        addr = nfsv4_1_file_layout_ds_addr4(stripe_indices, netaddrs)
+        addr = type4.nfsv4_1_file_layout_ds_addr4(stripe_indices, netaddrs)
         p = NFS4Packer()
         p.pack_nfsv4_1_file_layout_ds_addr4(addr)
         return p.get_buffer()
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 11/17] 4.1 server: move nfs4_ops.py to nfs_ops.py
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (8 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 10/17] dataserver: don't import * from nfs4 specific mods Weston Andros Adamson
@ 2014-06-04 21:01 ` Weston Andros Adamson
  2014-06-04 21:02 ` [PATCH pynfs 12/17] add mntv3, portmapv2 and nfsv3 .x files Weston Andros Adamson
                   ` (7 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:01 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Also replace ugly exec & inspect code to just define a class with
a __getattr__ switch.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 32 ++++++++++---------
 nfs4.1/fs.py         |  2 --
 nfs4.1/nfs4_ops.py   | 61 -----------------------------------
 nfs4.1/nfs4client.py | 16 +++++-----
 nfs4.1/nfs4lib.py    |  8 +++--
 nfs4.1/nfs4state.py  |  8 +++--
 nfs4.1/nfs_ops.py    | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 125 insertions(+), 91 deletions(-)
 delete mode 100644 nfs4.1/nfs4_ops.py
 create mode 100644 nfs4.1/nfs_ops.py

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index a825615..c73e195 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -8,11 +8,13 @@ import logging
 import nfs4client
 import hashlib
 import sys
-import nfs4_ops as op
+import nfs_ops
 import socket
 
 log = logging.getLogger("Dataserver Manager")
 
+op4 = nfs_ops.NFS4ops()
+
 class DataServer41(object):
     def __init__(self, server, port, path, flavor=rpc.AUTH_SYS, active=True, mdsds=True, multipath_servers=None, summary=None):
         self.mdsds = mdsds
@@ -121,12 +123,12 @@ class DataServer41(object):
                                exceptions=[const4.NFS4ERR_NOENT])
             if res.status == const4.NFS4ERR_NOENT:
                 cr_ops = nfs4lib.use_obj(existing_path[:-1]) + \
-                    [op.create(kind, comp, attrs)]
+                    [op4.create(kind, comp, attrs)]
                 self._execute(cr_ops)
-        res = self._execute(nfs4lib.use_obj(self.path) + [op.getfh()])
+        res = self._execute(nfs4lib.use_obj(self.path) + [op4.getfh()])
         self.path_fh = res.resarray[-1].object
         need = const4.ACCESS4_READ | const4.ACCESS4_LOOKUP | const4.ACCESS4_MODIFY | const4.ACCESS4_EXTEND
-        res = self._execute(nfs4lib.use_obj(self.path_fh) + [op.access(need)])
+        res = self._execute(nfs4lib.use_obj(self.path_fh) + [op4.access(need)])
         if res.resarray[-1].access != need:
             raise RuntimeError
         # XXX clean DS directory
@@ -143,10 +145,10 @@ class DataServer41(object):
         while True:
             if mds_fh in self.filehandles:
                 return
-            open_op = op.open(seqid, access, deny,
+            open_op = op4.open(seqid, access, deny,
                               type4.open_owner4(self.sess.client.clientid, owner),
                               openflag, type4.open_claim4(const4.CLAIM_NULL, name))
-            res = self._execute(nfs4lib.use_obj(self.path_fh) + [open_op, op.getfh()], exceptions=[const4.NFS4ERR_EXIST])
+            res = self._execute(nfs4lib.use_obj(self.path_fh) + [open_op, op4.getfh()], exceptions=[const4.NFS4ERR_EXIST])
             if res.status == const4.NFS4_OK:
                  ds_fh = res.resarray[-1].opgetfh.resok4.object
                  ds_openstateid = type4.stateid4(0, res.resarray[-2].stateid.other)
@@ -161,33 +163,33 @@ class DataServer41(object):
         """close the given file"""
         seqid=0 #FIXME: seqid must be !=0
         fh, stateid = self.filehandles[mds_fh]
-        ops = [op.putfh(fh)] + [op.close(seqid, stateid)]
+        ops = [op4.putfh(fh)] + [op4.close(seqid, stateid)]
         res = self._execute(ops)
         # ignoring return
         del self.filehandles[mds_fh]
 
     def read(self, fh, pos, count):
-        ops = [op.putfh(fh),
-               op.read(nfs4lib.state00, pos, count)]
+        ops = [op4.putfh(fh),
+               op4.read(nfs4lib.state00, pos, count)]
         # There are all sorts of error handling issues here
         res = self._execute(ops)
         data = res.resarray[-1].data
         return data
 
     def write(self, fh, pos, data):
-        ops = [op.putfh(fh),
-               op.write(nfs4lib.state00, pos, const4.FILE_SYNC4, data)]
+        ops = [op4.putfh(fh),
+               op4.write(nfs4lib.state00, pos, const4.FILE_SYNC4, data)]
         # There are all sorts of error handling issues here
         res = self._execute(ops)
 
     def truncate(self, fh, size):
-        ops = [op.putfh(fh),
-               op.setattr(nfs4lib.state00, {const4.FATTR4_SIZE: size})]
+        ops = [op4.putfh(fh),
+               op4.setattr(nfs4lib.state00, {const4.FATTR4_SIZE: size})]
         res = self._execute(ops)
 
     def get_size(self, fh):
-        ops = [op.putfh(fh),
-               op.getattr(1L << const4.FATTR4_SIZE)]
+        ops = [op4.putfh(fh),
+               op4.getattr(1L << const4.FATTR4_SIZE)]
         res = self._execute(ops)
         attrdict = res.resarray[-1].obj_attributes
         return attrdict.get(const4.FATTR4_SIZE, 0)
diff --git a/nfs4.1/fs.py b/nfs4.1/fs.py
index 8fc49ef..8947014 100644
--- a/nfs4.1/fs.py
+++ b/nfs4.1/fs.py
@@ -1557,8 +1557,6 @@ class FileLayoutFile(object): # XXX This should inherit from fs_base.py
         vol = FilelayoutVolWrapper(self._obj, device.list[index])
         return vol, v_pos, remaining
 
-import nfs4_ops as op
-
 class FilelayoutVolWrapper(object):
     def __init__(self, obj, dataserver):
         self._obj = obj
diff --git a/nfs4.1/nfs4_ops.py b/nfs4.1/nfs4_ops.py
deleted file mode 100644
index 35a10ca..0000000
--- a/nfs4.1/nfs4_ops.py
+++ /dev/null
@@ -1,61 +0,0 @@
-"""For each OP_<NAME> in nfs_argop4 and nfs_cb_argop4, create a function
-<name>() that returns the appropriate *_argop4 structure, hiding
-this routine packing from the user.
-"""
-import xdrdef.nfs4_type as _type
-import xdrdef.nfs4_const as _const
-
-# This string is our general function template
-code = """\
-def %(funct_name)s(%(funct_args)s):
-    %(create_args)s
-    return _type.%(argop)s(_const.OP_%(enum_name)s, %(set_args)s)
-"""
-
-def _mappings():
-    return _pull_argops(_const.nfs_opnum4) + _pull_argops(_const.nfs_cb_opnum4)
-
-def _pull_argops(op_dict):
-    """ For each entry in op_dict, create an appropriate dictionary that can
-        be used to fill the 'code' template.
-    """
-    import inspect
-    out = []
-    keys = op_dict.keys()
-    keys.sort() # Not necessary, but makes scanning the printout easier
-    for k in keys:
-        # Create a dictionary that will be used to fill the 'code' template
-        d = {}
-        d["enum_name"] = enum_name = op_dict[k][3:] # <NAME>
-        d["funct_name"] = "%s" % enum_name.lower() # <name>
-        class_name = "%s4args" % enum_name
-        klass = getattr(_type, class_name, None)
-        if klass is None:
-            # This operation takes no arguments
-            d["funct_args"] = d["create_args"] = d["set_args"] = ""
-        else:
-            if type(klass) is dict:
-                arg_list = "enum_value"
-                d["create_args"] = "args = enum_value"
-            else:
-                arg_list = ", ".join(inspect.getargspec(klass.__init__)[0][1:])
-                d["create_args"] = "args = _type.%s(%s)" % (class_name, arg_list)
-            d["funct_args"] = arg_list
-            if enum_name.startswith("CB_"):
-                d["set_args"] = "opcb%s=args" % enum_name.lower()[3:]
-            else:
-                d["set_args"] = "op%s=args" % enum_name.lower()
-        if enum_name.startswith("CB_"):
-            d["argop"] = "nfs_cb_argop4"
-        else:
-            d["argop"] = "nfs_argop4"
-        out.append(d)
-    return out
-
-if __name__ == "__main__":
-    for _d in _mappings():
-        print code % _d
-else:
-    for _d in _mappings():
-        exec code % _d
-    
diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
index f5d2006..263f37d 100644
--- a/nfs4.1/nfs4client.py
+++ b/nfs4.1/nfs4client.py
@@ -5,7 +5,7 @@ from nfs4lib import NFS4Error, NFS4Replay, inc_u32
 from xdrdef.nfs4_type import *
 from xdrdef.nfs4_const import *
 from xdrdef.sctrl_pack import SCTRLPacker, SCTRLUnpacker
-import nfs4_ops as op
+import nfs_ops
 import time, struct
 import threading
 import hmac
@@ -20,6 +20,8 @@ logging.basicConfig(level=logging.INFO,
                     format="%(levelname)-7s:%(name)s:%(message)s")
 log_cb = logging.getLogger("nfs.client.cb")
 
+op4 = nfs_ops.NFS4ops()
+
 class NFS4Client(rpc.Client, rpc.Server):
     def __init__(self, host='localhost', port=2049, minorversion=1, ctrl_proc=16, summary=None):
         rpc.Client.__init__(self, 100003, 4)
@@ -275,7 +277,7 @@ class NFS4Client(rpc.Client, rpc.Server):
         owner = client_owner4(verf, name)
         if protect is None:
             protect = state_protect4_a(SP4_NONE)
-        res = self.compound([op.exchange_id(owner, flags, protect,
+        res = self.compound([op4.exchange_id(owner, flags, protect,
                                             [self.impl_id])],
                             cred)
         nfs4lib.check(res, expect)
@@ -287,7 +289,7 @@ class NFS4Client(rpc.Client, rpc.Server):
     def new_client_session(self, name, flags=0, sec=None):
         c = self.new_client(name, flags=flags)
         s = c.create_session(sec=sec)
-        s.compound([op.reclaim_complete(FALSE)])
+        s.compound([op4.reclaim_complete(FALSE)])
         return s
 
 class ClientStateProtection(object):
@@ -339,7 +341,7 @@ class ClientRecord(object):
         if prog is None:
             prog = self.c.prog
         for item in xrange(max_retries):
-            res = self.c.compound([op.create_session(self.clientid, self.seqid,
+            res = self.c.compound([op4.create_session(self.clientid, self.seqid,
                                                  flags,
                                                  fore_attrs, back_attrs,
                                                  prog, sec)],
@@ -362,7 +364,7 @@ class ClientRecord(object):
         self.seqid = inc_u32(csr.csr_sequence) # XXX Do we need to check this?
         sess = SessionRecord(csr, self)
         self.c.sessions[sess.sessionid] = sess
-        sess.compound([op.reclaim_complete(FALSE)])
+        sess.compound([op4.reclaim_complete(FALSE)])
         return sess
 
     def _cb_hook(self, prefix, opname, funct):
@@ -430,7 +432,7 @@ class SessionRecord(object):
             raise RuntimeError
             slot = self.fore_channel.slots[slot]
         # STUB, need to properly set highest
-        return op.sequence(self.sessionid, slot.get_seqid(seq_delta),
+        return op4.sequence(self.sessionid, slot.get_seqid(seq_delta),
                            slot.id, slot.id, cache_this)
 
     def set_ssv(self, ssv=None, *args, **kwargs):
@@ -443,7 +445,7 @@ class SessionRecord(object):
         p = nfs4lib.FancyNFS4Packer()
         p.pack_SEQUENCE4args(seq_op.opsequence)
         digest =  protect.context.hmac(p.get_buffer(), SSV4_SUBKEY_MIC_I2T)
-        ssv_op = op.set_ssv(ssv, digest)
+        ssv_op = op4.set_ssv(ssv, digest)
         res = self.c.compound([seq_op, ssv_op], *args, **kwargs)
         # STUB - do some checking
         protect.context.set_ssv(ssv)
diff --git a/nfs4.1/nfs4lib.py b/nfs4.1/nfs4lib.py
index 116324a..02352e1 100644
--- a/nfs4.1/nfs4lib.py
+++ b/nfs4.1/nfs4lib.py
@@ -3,7 +3,7 @@ import rpc
 import xdrdef.nfs4_const
 from xdrdef.nfs4_pack import NFS4Packer, NFS4Unpacker
 import xdrdef.nfs4_type
-import nfs4_ops as op
+import nfs_ops
 import time
 import collections
 import hmac
@@ -30,6 +30,8 @@ state01 = xdrdef.nfs4_type.stateid4(1, "\0" * 12)
 
 import hashlib # Note this requires 2.5 or higher
 
+op4 = nfs_ops.NFS4ops()
+
 # Note that all the oid strings have tag and length bytes prepended, as
 # per description of sec_oid4 in draft26 sect 3.2
 
@@ -626,9 +628,9 @@ def use_obj(file):
     if file is None or file == [None]:
         return []
     elif type(file) is str:
-        return [op.putfh(file)]
+        return [op4.putfh(file)]
     else:
-        return [op.putrootfh()] + [op.lookup(comp) for comp in file]
+        return [op4.putrootfh()] + [op4.lookup(comp) for comp in file]
 
 ###############################################
 # Attribute information
diff --git a/nfs4.1/nfs4state.py b/nfs4.1/nfs4state.py
index 2f3cd59..2214c0d 100644
--- a/nfs4.1/nfs4state.py
+++ b/nfs4.1/nfs4state.py
@@ -8,12 +8,14 @@ from nfs4lib import NFS4Error
 #from xdrdef.nfs4_type import stateid4
 from xdrdef.nfs4_type import *
 from xdrdef.nfs4_const import *
-import nfs4_ops as op
+import nfs_ops
 import rpc
 import logging
 
 log = logging.getLogger("nfs.server.state")
 
+op4 = nfs_ops.NFS4ops()
+
 POSIXLOCK = False
 
 SHARE, BYTE, DELEG, LAYOUT, ANON = range(5) # State types
@@ -748,9 +750,9 @@ class DelegEntry(StateTableEntry):
         # ANSWER - we care about self.status, which can be set to 
         # INVALID anytime by deleg_return
         slot = session.channel_back.choose_slot()
-        seq_op = op.cb_sequence(session.sessionid, slot.get_seqid(),
+        seq_op = op4.cb_sequence(session.sessionid, slot.get_seqid(),
                                 slot.id, slot.id, True, []) # STUB
-        recall_op = op.cb_recall(self.get_id(cb=True), False, self.file.fh)
+        recall_op = op4.cb_recall(self.get_id(cb=True), False, self.file.fh)
         if self.invalid:
             # Race here doesn't matter, but would like to avoid the
             # RPC if possible.
diff --git a/nfs4.1/nfs_ops.py b/nfs4.1/nfs_ops.py
new file mode 100644
index 0000000..0753716
--- /dev/null
+++ b/nfs4.1/nfs_ops.py
@@ -0,0 +1,89 @@
+"""For each OP_<NAME> in nfs_argop4 and nfs_cb_argop4, create a function
+<name>() that returns the appropriate *_argop4 structure, hiding
+this routine packing from the user.
+"""
+
+from xdrdef import nfs4_type
+from xdrdef import nfs4_const
+
+from xdrdef import nfs3_type
+from xdrdef import nfs3_const
+
+def nfs4_op_names():
+    skip = len('OP_')
+    ops = [ x.lower()[skip:] for x in nfs4_const.nfs_opnum4.values() ]
+    ops.extend([ x.lower()[skip:] for x in nfs4_const.nfs_cb_opnum4.values()])
+    return ops
+
+def nfs3_proc_names():
+    pre = 'NFSPROC3_'
+    skip = len(pre)
+    procs = [ x.lower()[skip:] for x in dir(nfs3_const) if x.startswith(pre) ]
+    return procs
+
+class NFSops:
+    def __init__(self, is_v4):
+        self._is_v4 = is_v4
+        if is_v4:
+            self._op_names = nfs4_op_names()
+            self._type = nfs4_type
+            self._const = nfs4_const
+            self._args_suffix = '4args'
+            self._op_prefix = 'OP_'
+        else:
+            self._op_names = nfs3_proc_names()
+            self._type = nfs3_type
+            self._const = nfs3_const
+            self._args_suffix = '3args'
+            self._op_prefix = 'NFSPROC3_'
+
+    def __getattr__(self, attrname):
+        if attrname in self._op_names:
+            return lambda *args: self._handle_op(attrname, args)
+
+    def _handle_op(self, opname, args):
+        enum_name = opname.upper()
+
+        # RPC "args" class to create
+        class_name = "%s%s" % (enum_name, self._args_suffix)
+        klass = getattr(self._type, class_name, None)
+
+        if self._is_v4:
+            # stuff class into argop
+
+            # args to pass to argop __init__
+            opnum = getattr(self._const, self._op_prefix + enum_name)
+            kwargs = {}
+
+            if klass:
+                # otherwise it takes no arguments
+                if type(klass) is dict:
+                    assert len(args) == 1
+                    arg = args[0]
+                else:
+                    arg = klass(*args)
+
+                if enum_name.startswith("CB_"):
+                    kwargs['opcb%s' % enum_name.lower()] = arg
+                else:
+                    kwargs['op%s' % enum_name.lower()] = arg
+
+            if enum_name.startswith("CB_"):
+                argop = self._type.nfs_cb_argop4
+            else:
+                argop = self._type.nfs_argop4
+
+            return argop(opnum, **kwargs)
+
+        else:
+            # for v3 just return an instance
+            return klass(*args)
+
+class NFS3ops(NFSops):
+    def __init__(self):
+        NFSops.__init__(self, False)
+
+class NFS4ops(NFSops):
+    def __init__(self):
+        NFSops.__init__(self, True)
+
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 12/17] add mntv3, portmapv2 and nfsv3 .x files
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (9 preceding siblings ...)
  2014-06-04 21:01 ` [PATCH pynfs 11/17] 4.1 server: move nfs4_ops.py to nfs_ops.py Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
  2014-06-05  2:34   ` J. Bruce Fields
  2014-06-04 21:02 ` [PATCH pynfs 13/17] dataserver: separate generic and 4.1 code Weston Andros Adamson
                   ` (6 subsequent siblings)
  17 siblings, 1 reply; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

mntv3 and nfsv3 .x files made from IETF specs by Tom Haynes
<thomas.haynes@primarydata.com>.

portmap2 .x files made from IETF specs by Weston Andros Adamson
<dros@primarydata.com>.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/xdrdef/mnt3.x    |  68 ++++
 nfs4.1/xdrdef/nfs3.x    | 818 ++++++++++++++++++++++++++++++++++++++++++++++++
 nfs4.1/xdrdef/portmap.x |  51 +++
 3 files changed, 937 insertions(+)
 create mode 100644 nfs4.1/xdrdef/mnt3.x
 create mode 100644 nfs4.1/xdrdef/nfs3.x
 create mode 100644 nfs4.1/xdrdef/portmap.x

diff --git a/nfs4.1/xdrdef/mnt3.x b/nfs4.1/xdrdef/mnt3.x
new file mode 100644
index 0000000..b632322
--- /dev/null
+++ b/nfs4.1/xdrdef/mnt3.x
@@ -0,0 +1,68 @@
+const MNTPATHLEN = 1024;   /* Maximum bytes in a path name */
+const MNTNAMLEN = 255;     /* Maximum bytes in a name */
+const FHSIZE3 = 64;        /* Maximum bytes in a V3 file handle */
+
+typedef opaque  fhandle3 <FHSIZE3>;
+typedef string  dirpath <MNTPATHLEN>;
+typedef string  name <MNTNAMLEN>;
+
+struct mountres3_ok {
+        fhandle3        fhandle;
+        int             auth_flavors <>;
+};
+
+union mountres3 switch (mountstat3 fhs_status) {
+case MNT3_OK:
+        mountres3_ok mountinfo;
+default:
+        void;
+};
+
+struct mountbody {
+        name            ml_hostname;
+        dirpath         ml_directory;
+        mountlist       ml_next;
+};
+
+typedef mountbody *mountlist;
+
+
+struct groupnode {
+        name            gr_name;
+        groups          gr_next;
+};
+
+typedef groupnode *groups;
+
+
+struct exportnode {
+        dirpath         ex_dir;
+        groups          ex_groups;
+        exports         ex_next;
+};
+
+typedef exportnode *exports;
+
+enum mountstat3 {
+        MNT3_OK = 0,                     /* no error */
+        MNT3ERR_PERM = 1,                /* Not owner */
+        MNT3ERR_NOENT = 2,               /* No such file or directory */
+        MNT3ERR_IO = 5,                  /* I/O error */
+        MNT3ERR_ACCES = 13,              /* Permission denied */
+        MNT3ERR_NOTDIR = 20,             /* Not a directory */
+        MNT3ERR_INVAL = 22,              /* Invalid argument */
+        MNT3ERR_NAMETOOLONG = 63,        /* Filename too long */
+        MNT3ERR_NOTSUPP = 10004,         /* Operation not supported */
+        MNT3ERR_SERVERFAULT = 10006      /* A failure on the server */
+};
+
+program MOUNT_PROGRAM {
+        version MOUNT_V3 {
+                void            MOUNTPROC3_NULL(void) = 0;
+                mountres3       MOUNTPROC3_MNT(dirpath) = 1;
+                mountlist       MOUNTPROC3_DUMP(void) = 2;
+                void            MOUNTPROC3_UMNT(dirpath) = 3;
+                void            MOUNTPROC3_UMNTALL(void) = 4;
+                exports         MOUNTPROC3_EXPORT(void) = 5;
+        } = 3;
+} = 100005;
diff --git a/nfs4.1/xdrdef/nfs3.x b/nfs4.1/xdrdef/nfs3.x
new file mode 100644
index 0000000..ddd1395
--- /dev/null
+++ b/nfs4.1/xdrdef/nfs3.x
@@ -0,0 +1,818 @@
+const NFS3_FHSIZE = 64;        /* Maximum bytes in a V3 file handle */
+
+const FALSE = 0;
+const TRUE = 1;
+
+const NFS3_COOKIEVERFSIZE = 8;
+const NFS3_CREATEVERFSIZE = 8;
+const NFS3_WRITEVERFSIZE = 8;
+
+typedef unsigned hyper uint64;
+typedef hyper   int64;
+typedef unsigned int uint32;
+typedef int    int32;
+typedef string  filename3 <>;
+typedef string  nfspath3 <>;
+typedef uint64  fileid3;
+typedef uint64  cookie3;
+typedef opaque  cookieverf3[NFS3_COOKIEVERFSIZE];
+typedef opaque  createverf3[NFS3_CREATEVERFSIZE];
+typedef opaque  writeverf3[NFS3_WRITEVERFSIZE];
+typedef uint32  uid3;
+typedef uint32  gid3;
+typedef uint64  size3;
+typedef uint64  offset3;
+typedef uint32  mode3;
+typedef uint32  count3;
+
+enum nfsstat3 {
+        NFS3_OK = 0,
+        NFS3ERR_PERM = 1,
+        NFS3ERR_NOENT = 2,
+        NFS3ERR_IO = 5,
+        NFS3ERR_NXIO = 6,
+        NFS3ERR_ACCES = 13,
+        NFS3ERR_EXIST = 17,
+        NFS3ERR_XDEV = 18,
+        NFS3ERR_NODEV = 19,
+        NFS3ERR_NOTDIR = 20,
+        NFS3ERR_ISDIR = 21,
+        NFS3ERR_INVAL = 22,
+        NFS3ERR_FBIG = 27,
+        NFS3ERR_NOSPC = 28,
+        NFS3ERR_ROFS = 30,
+        NFS3ERR_MLINK = 31,
+        NFS3ERR_NAMETOOLONG = 63,
+        NFS3ERR_NOTEMPTY = 66,
+        NFS3ERR_DQUOT = 69,
+        NFS3ERR_STALE = 70,
+        NFS3ERR_REMOTE = 71,
+        NFS3ERR_BADHANDLE = 10001,
+        NFS3ERR_NOT_SYNC = 10002,
+        NFS3ERR_BAD_COOKIE = 10003,
+        NFS3ERR_NOTSUPP = 10004,
+        NFS3ERR_TOOSMALL = 10005,
+        NFS3ERR_SERVERFAULT = 10006,
+        NFS3ERR_BADTYPE = 10007,
+        NFS3ERR_JUKEBOX = 10008
+};
+
+enum ftype3 {
+        NF3REG = 1,
+        NF3DIR = 2,
+        NF3BLK = 3,
+        NF3CHR = 4,
+        NF3LNK = 5,
+        NF3SOCK = 6,
+        NF3FIFO = 7
+};
+
+struct specdata3 {
+        uint32          specdata1;
+        uint32          specdata2;
+};
+
+struct nfs_fh3 {
+        opaque          data <NFS3_FHSIZE>;
+};
+
+struct nfstime3 {
+        uint32          seconds;
+        uint32          nseconds;
+};
+
+struct fattr3 {
+        ftype3          type;
+        mode3           mode;
+        uint32          nlink;
+        uid3            uid;
+        gid3            gid;
+        size3           size;
+        size3           used;
+        specdata3       rdev;
+        uint64          fsid;
+        fileid3         fileid;
+        nfstime3        atime;
+        nfstime3        mtime;
+        nfstime3        ctime;
+};
+
+union post_op_attr switch (bool attributes_follow) {
+case TRUE:
+        fattr3 attributes;
+case FALSE:
+        void;
+};
+
+struct wcc_attr {
+        size3           size;
+        nfstime3        mtime;
+        nfstime3        ctime;
+};
+
+union pre_op_attr switch (bool attributes_follow) {
+case TRUE:
+        wcc_attr attributes;
+case FALSE:
+        void;
+};
+
+struct wcc_data {
+        pre_op_attr     before;
+        post_op_attr    after;
+};
+
+union post_op_fh3 switch (bool handle_follows) {
+case TRUE:
+        nfs_fh3 handle;
+case FALSE:
+        void;
+};
+
+enum time_how {
+        DONT_CHANGE = 0,
+        SET_TO_SERVER_TIME = 1,
+        SET_TO_CLIENT_TIME = 2
+};
+
+union set_mode3 switch (bool set_it) {
+case TRUE:
+        mode3 mode;
+default:
+        void;
+};
+
+union set_uid3 switch (bool set_it) {
+case TRUE:
+        uid3 uid;
+default:
+        void;
+};
+
+union set_gid3 switch (bool set_it) {
+case TRUE:
+        gid3 gid;
+default:
+        void;
+};
+
+union set_size3 switch (bool set_it) {
+case TRUE:
+        size3 size;
+default:
+        void;
+};
+
+union set_atime switch (time_how set_it) {
+case SET_TO_CLIENT_TIME:
+        nfstime3 atime;
+default:
+        void;
+};
+
+union set_mtime switch (time_how set_it) {
+case SET_TO_CLIENT_TIME:
+        nfstime3 mtime;
+default:
+        void;
+};
+
+struct sattr3 {
+        set_mode3       mode;
+        set_uid3        uid;
+        set_gid3        gid;
+        set_size3       size;
+        set_atime       atime;
+        set_mtime       mtime;
+};
+
+struct diropargs3 {
+        nfs_fh3         dir;
+        filename3       name;
+};
+
+struct GETATTR3args {
+        nfs_fh3         object;
+};
+
+struct GETATTR3resok {
+        fattr3          obj_attributes;
+};
+
+union GETATTR3res switch (nfsstat3 status) {
+case NFS3_OK:
+        GETATTR3resok resok;
+default:
+        void;
+};
+
+union sattrguard3 switch (bool check) {
+case TRUE:
+        nfstime3 obj_ctime;
+case FALSE:
+        void;
+};
+
+struct SETATTR3args {
+        nfs_fh3         object;
+        sattr3          new_attributes;
+        sattrguard3     guard;
+};
+
+struct SETATTR3resok {
+        wcc_data        obj_wcc;
+};
+
+struct SETATTR3resfail {
+        wcc_data        obj_wcc;
+};
+
+union SETATTR3res switch (nfsstat3 status) {
+case NFS3_OK:
+        SETATTR3resok resok;
+default:
+        SETATTR3resfail resfail;
+};
+
+struct LOOKUP3args {
+        diropargs3      what;
+};
+
+struct LOOKUP3resok {
+        nfs_fh3         object;
+        post_op_attr    obj_attributes;
+        post_op_attr    dir_attributes;
+};
+
+struct LOOKUP3resfail {
+        post_op_attr    dir_attributes;
+};
+
+union LOOKUP3res switch (nfsstat3 status) {
+case NFS3_OK:
+        LOOKUP3resok resok;
+default:
+        LOOKUP3resfail resfail;
+};
+
+const           ACCESS3_READ = 0x0001;
+const           ACCESS3_LOOKUP = 0x0002;
+const           ACCESS3_MODIFY = 0x0004;
+const           ACCESS3_EXTEND = 0x0008;
+const           ACCESS3_DELETE = 0x0010;
+const           ACCESS3_EXECUTE = 0x0020;
+
+struct ACCESS3args {
+        nfs_fh3         object;
+        uint32          access;
+};
+
+struct ACCESS3resok {
+        post_op_attr    obj_attributes;
+        uint32          access;
+};
+
+struct ACCESS3resfail {
+        post_op_attr    obj_attributes;
+};
+
+union ACCESS3res switch (nfsstat3 status) {
+case NFS3_OK:
+        ACCESS3resok resok;
+default:
+        ACCESS3resfail resfail;
+};
+
+struct READLINK3args {
+        nfs_fh3         symlink;
+};
+
+struct READLINK3resok {
+        post_op_attr    symlink_attributes;
+        nfspath3        data;
+};
+
+struct READLINK3resfail {
+        post_op_attr    symlink_attributes;
+};
+
+union READLINK3res switch (nfsstat3 status) {
+case NFS3_OK:
+        READLINK3resok resok;
+default:
+        READLINK3resfail resfail;
+};
+
+struct READ3args {
+        nfs_fh3         file;
+        offset3         offset;
+        count3          count;
+};
+
+struct READ3resok {
+        post_op_attr    file_attributes;
+        count3          count;
+        bool            eof;
+        opaque          data <>;
+};
+
+struct READ3resfail {
+        post_op_attr    file_attributes;
+};
+
+union READ3res switch (nfsstat3 status) {
+case NFS3_OK:
+        READ3resok resok;
+default:
+        READ3resfail resfail;
+};
+
+enum stable_how {
+        UNSTABLE = 0,
+        DATA_SYNC = 1,
+        FILE_SYNC = 2
+};
+
+struct WRITE3args {
+        nfs_fh3         file;
+        offset3         offset;
+        count3          count;
+        stable_how      stable;
+        opaque          data <>;
+};
+
+struct WRITE3resok {
+        wcc_data        file_wcc;
+        count3          count;
+        stable_how      committed;
+        writeverf3      verf;
+};
+
+struct WRITE3resfail {
+        wcc_data        file_wcc;
+};
+
+union WRITE3res switch (nfsstat3 status) {
+case NFS3_OK:
+        WRITE3resok resok;
+default:
+        WRITE3resfail resfail;
+};
+
+enum createmode3 {
+        UNCHECKED = 0,
+        GUARDED = 1,
+        EXCLUSIVE = 2
+};
+
+union createhow3 switch (createmode3 mode) {
+case UNCHECKED:
+case GUARDED:
+        sattr3 obj_attributes;
+case EXCLUSIVE:
+        createverf3 verf;
+};
+
+struct CREATE3args {
+        diropargs3      where;
+        createhow3      how;
+};
+
+struct CREATE3resok {
+        post_op_fh3     obj;
+        post_op_attr    obj_attributes;
+        wcc_data        dir_wcc;
+};
+
+struct CREATE3resfail {
+        wcc_data        dir_wcc;
+};
+
+union CREATE3res switch (nfsstat3 status) {
+case NFS3_OK:
+        CREATE3resok resok;
+default:
+        CREATE3resfail resfail;
+};
+
+struct MKDIR3args {
+        diropargs3      where;
+        sattr3          attributes;
+};
+
+struct MKDIR3resok {
+        post_op_fh3     obj;
+        post_op_attr    obj_attributes;
+        wcc_data        dir_wcc;
+};
+
+struct MKDIR3resfail {
+        wcc_data        dir_wcc;
+};
+
+union MKDIR3res switch (nfsstat3 status) {
+case NFS3_OK:
+        MKDIR3resok resok;
+default:
+        MKDIR3resfail resfail;
+};
+
+struct symlinkdata3 {
+        sattr3          symlink_attributes;
+        nfspath3        symlink_data;
+};
+
+struct SYMLINK3args {
+        diropargs3      where;
+        symlinkdata3    symlink;
+};
+
+struct SYMLINK3resok {
+        post_op_fh3     obj;
+        post_op_attr    obj_attributes;
+        wcc_data        dir_wcc;
+};
+
+struct SYMLINK3resfail {
+        wcc_data        dir_wcc;
+};
+
+union SYMLINK3res switch (nfsstat3 status) {
+case NFS3_OK:
+        SYMLINK3resok resok;
+default:
+        SYMLINK3resfail resfail;
+};
+
+struct devicedata3 {
+        sattr3          dev_attributes;
+        specdata3       spec;
+};
+
+union mknoddata3 switch (ftype3 type) {
+case NF3CHR:
+case NF3BLK:
+        devicedata3 device;
+case NF3SOCK:
+case NF3FIFO:
+        sattr3 pipe_attributes;
+default:
+        void;
+};
+
+struct MKNOD3args {
+        diropargs3      where;
+        mknoddata3      what;
+};
+
+struct MKNOD3resok {
+        post_op_fh3     obj;
+        post_op_attr    obj_attributes;
+        wcc_data        dir_wcc;
+};
+
+struct MKNOD3resfail {
+        wcc_data        dir_wcc;
+};
+
+union MKNOD3res switch (nfsstat3 status) {
+case NFS3_OK:
+        MKNOD3resok resok;
+default:
+        MKNOD3resfail resfail;
+};
+
+struct REMOVE3args {
+        diropargs3      object;
+};
+
+struct REMOVE3resok {
+        wcc_data        dir_wcc;
+};
+
+struct REMOVE3resfail {
+        wcc_data        dir_wcc;
+};
+
+union REMOVE3res switch (nfsstat3 status) {
+case NFS3_OK:
+        REMOVE3resok resok;
+default:
+        REMOVE3resfail resfail;
+};
+
+struct RMDIR3args {
+        diropargs3      object;
+};
+
+struct RMDIR3resok {
+        wcc_data        dir_wcc;
+};
+
+struct RMDIR3resfail {
+        wcc_data        dir_wcc;
+};
+
+union RMDIR3res switch (nfsstat3 status) {
+case NFS3_OK:
+        RMDIR3resok resok;
+default:
+        RMDIR3resfail resfail;
+};
+
+struct RENAME3args {
+        diropargs3      fromfile;
+        diropargs3      tofile;
+};
+
+struct RENAME3resok {
+        wcc_data        fromdir_wcc;
+        wcc_data        todir_wcc;
+};
+
+struct RENAME3resfail {
+        wcc_data        fromdir_wcc;
+        wcc_data        todir_wcc;
+};
+
+union RENAME3res switch (nfsstat3 status) {
+case NFS3_OK:
+        RENAME3resok resok;
+default:
+        RENAME3resfail resfail;
+};
+
+struct LINK3args {
+        nfs_fh3         file;
+        diropargs3      link;
+};
+
+struct LINK3resok {
+        post_op_attr    file_attributes;
+        wcc_data        linkdir_wcc;
+};
+
+struct LINK3resfail {
+        post_op_attr    file_attributes;
+        wcc_data        linkdir_wcc;
+};
+
+union LINK3res switch (nfsstat3 status) {
+case NFS3_OK:
+        LINK3resok resok;
+default:
+        LINK3resfail resfail;
+};
+
+struct READDIR3args {
+        nfs_fh3         dir;
+        cookie3         cookie;
+        cookieverf3     cookieverf;
+        count3          count;
+};
+
+struct entry3 {
+        fileid3         fileid;
+        filename3       name;
+        cookie3         cookie;
+        entry3         *nextentry;
+};
+
+struct dirlist3 {
+        entry3         *entries;
+        bool            eof;
+};
+
+struct READDIR3resok {
+        post_op_attr    dir_attributes;
+        cookieverf3     cookieverf;
+        dirlist3        reply;
+};
+
+struct READDIR3resfail {
+        post_op_attr    dir_attributes;
+};
+
+union READDIR3res switch (nfsstat3 status) {
+case NFS3_OK:
+        READDIR3resok resok;
+default:
+        READDIR3resfail resfail;
+};
+
+struct READDIRPLUS3args {
+        nfs_fh3         dir;
+        cookie3         cookie;
+        cookieverf3     cookieverf;
+        count3          dircount;
+        count3          maxcount;
+};
+
+struct entryplus3 {
+        fileid3         fileid;
+        filename3       name;
+        cookie3         cookie;
+        post_op_attr    name_attributes;
+        post_op_fh3     name_handle;
+        entryplus3     *nextentry;
+};
+
+struct dirlistplus3 {
+        entryplus3     *entries;
+        bool            eof;
+};
+
+struct READDIRPLUS3resok {
+        post_op_attr    dir_attributes;
+        cookieverf3     cookieverf;
+        dirlistplus3    reply;
+};
+
+struct READDIRPLUS3resfail {
+        post_op_attr    dir_attributes;
+};
+
+union READDIRPLUS3res switch (nfsstat3 status) {
+case NFS3_OK:
+        READDIRPLUS3resok resok;
+default:
+        READDIRPLUS3resfail resfail;
+};
+
+struct FSSTAT3args {
+        nfs_fh3         fsroot;
+};
+
+struct FSSTAT3resok {
+        post_op_attr    obj_attributes;
+        size3           tbytes;
+        size3           fbytes;
+        size3           abytes;
+        size3           tfiles;
+        size3           ffiles;
+        size3           afiles;
+        uint32          invarsec;
+};
+
+struct FSSTAT3resfail {
+        post_op_attr    obj_attributes;
+};
+
+union FSSTAT3res switch (nfsstat3 status) {
+case NFS3_OK:
+        FSSTAT3resok resok;
+default:
+        FSSTAT3resfail resfail;
+};
+
+const           FSF3_LINK = 0x0001;
+const           FSF3_SYMLINK = 0x0002;
+const           FSF3_HOMOGENEOUS = 0x0008;
+const           FSF3_CANSETTIME = 0x0010;
+
+struct FSINFOargs {
+        nfs_fh3         fsroot;
+};
+
+struct FSINFO3resok {
+        post_op_attr    obj_attributes;
+        uint32          rtmax;
+        uint32          rtpref;
+        uint32          rtmult;
+        uint32          wtmax;
+        uint32          wtpref;
+        uint32          wtmult;
+        uint32          dtpref;
+        size3           maxfilesize;
+        nfstime3        time_delta;
+        uint32          properties;
+};
+
+struct FSINFO3resfail {
+        post_op_attr    obj_attributes;
+};
+
+union FSINFO3res switch (nfsstat3 status) {
+case NFS3_OK:
+        FSINFO3resok resok;
+default:
+        FSINFO3resfail resfail;
+};
+
+struct PATHCONF3args {
+        nfs_fh3         object;
+};
+
+struct PATHCONF3resok {
+        post_op_attr    obj_attributes;
+        uint32          linkmax;
+        uint32          name_max;
+        bool            no_trunc;
+        bool            chown_restricted;
+        bool            case_insensitive;
+        bool            case_preserving;
+};
+
+struct PATHCONF3resfail {
+        post_op_attr    obj_attributes;
+};
+
+union PATHCONF3res switch (nfsstat3 status) {
+case NFS3_OK:
+        PATHCONF3resok resok;
+default:
+        PATHCONF3resfail resfail;
+};
+
+struct COMMIT3args {
+        nfs_fh3         file;
+        offset3         offset;
+        count3          count;
+};
+
+struct COMMIT3resok {
+        wcc_data        file_wcc;
+        writeverf3      verf;
+};
+
+struct COMMIT3resfail {
+        wcc_data        file_wcc;
+};
+
+union COMMIT3res switch (nfsstat3 status) {
+case NFS3_OK:
+        COMMIT3resok resok;
+default:
+        COMMIT3resfail resfail;
+};
+
+program         NFS_PROGRAM {
+        version         NFS_V3 {
+
+                void
+                NFSPROC3_NULL(void) = 0;
+
+                GETATTR3res
+                NFSPROC3_GETATTR(GETATTR3args) = 1;
+
+                SETATTR3res
+                NFSPROC3_SETATTR(SETATTR3args) = 2;
+
+                LOOKUP3res
+                NFSPROC3_LOOKUP(LOOKUP3args) = 3;
+
+                ACCESS3res
+                NFSPROC3_ACCESS(ACCESS3args) = 4;
+
+                READLINK3res
+                NFSPROC3_READLINK(READLINK3args) = 5;
+
+                READ3res
+                NFSPROC3_READ(READ3args) = 6;
+
+                WRITE3res
+                NFSPROC3_WRITE(WRITE3args) = 7;
+
+                CREATE3res
+                NFSPROC3_CREATE(CREATE3args) = 8;
+
+                MKDIR3res
+                NFSPROC3_MKDIR(MKDIR3args) = 9;
+
+                SYMLINK3res
+                NFSPROC3_SYMLINK(SYMLINK3args) = 10;
+
+                MKNOD3res
+                NFSPROC3_MKNOD(MKNOD3args) = 11;
+
+                REMOVE3res
+                NFSPROC3_REMOVE(REMOVE3args) = 12;
+
+                RMDIR3res
+                NFSPROC3_RMDIR(RMDIR3args) = 13;
+
+                RENAME3res
+                NFSPROC3_RENAME(RENAME3args) = 14;
+
+                LINK3res
+                NFSPROC3_LINK(LINK3args) = 15;
+
+                READDIR3res
+                NFSPROC3_READDIR(READDIR3args) = 16;
+
+                READDIRPLUS3res
+                NFSPROC3_READDIRPLUS(READDIRPLUS3args) = 17;
+
+                FSSTAT3res
+                NFSPROC3_FSSTAT(FSSTAT3args) = 18;
+
+                FSINFO3res
+                NFSPROC3_FSINFO(FSINFO3args) = 19;
+
+                PATHCONF3res
+                NFSPROC3_PATHCONF(PATHCONF3args) = 20;
+
+                COMMIT3res
+                NFSPROC3_COMMIT(COMMIT3args) = 21;
+        } = 3;
+} = 100003;
diff --git a/nfs4.1/xdrdef/portmap.x b/nfs4.1/xdrdef/portmap.x
new file mode 100644
index 0000000..1158d4e
--- /dev/null
+++ b/nfs4.1/xdrdef/portmap.x
@@ -0,0 +1,51 @@
+const PMAP_PORT = 111;      /* portmapper port number */
+
+struct mapping {
+	unsigned int prog;
+	unsigned int vers;
+	unsigned int prot;
+	unsigned int port;
+};
+
+const IPPROTO_TCP = 6;      /* protocol number for TCP/IP */
+const IPPROTO_UDP = 17;     /* protocol number for UDP/IP */
+
+struct pmaplist {
+	mapping map;
+	pmaplist next;
+};
+
+struct call_args {
+	unsigned int prog;
+	unsigned int vers;
+	unsigned int proc;
+	opaque args<>;
+};
+
+struct call_result {
+	unsigned int port;
+	opaque res<>;
+};
+
+program PMAP_PROG {
+	version PMAP_VERS {
+		void
+		PMAPPROC_NULL(void)         = 0;
+
+		bool
+		PMAPPROC_SET(mapping)       = 1;
+
+		bool
+		PMAPPROC_UNSET(mapping)     = 2;
+
+		unsigned int
+		PMAPPROC_GETPORT(mapping)   = 3;
+
+		pmaplist
+		PMAPPROC_DUMP(void)         = 4;
+
+		call_result
+		PMAPPROC_CALLIT(call_args)  = 5;
+	} = 2;
+} = 100000;
+
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 13/17] dataserver: separate generic and 4.1 code
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (10 preceding siblings ...)
  2014-06-04 21:02 ` [PATCH pynfs 12/17] add mntv3, portmapv2 and nfsv3 .x files Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
  2014-06-04 21:02 ` [PATCH pynfs 14/17] 4.1 server: add support for NFSv3 data servers Weston Andros Adamson
                   ` (5 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 99 ++++++++++++++++++++++++++++------------------------
 1 file changed, 54 insertions(+), 45 deletions(-)

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index c73e195..40364d4 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -15,7 +15,7 @@ log = logging.getLogger("Dataserver Manager")
 
 op4 = nfs_ops.NFS4ops()
 
-class DataServer41(object):
+class DataServer(object):
     def __init__(self, server, port, path, flavor=rpc.AUTH_SYS, active=True, mdsds=True, multipath_servers=None, summary=None):
         self.mdsds = mdsds
         self.server = server
@@ -43,28 +43,44 @@ class DataServer41(object):
         self.active = True
         if not self.mdsds:
             self.connect()
+            self.make_root()
 
     def down(self):
+        self.disconnect()
         self.active = False
 
-    def connect(self):
-        # only support root with AUTH_SYS for now
-        s1 = rpc.security.instance(rpc.AUTH_SYS)
-        self.cred1 = s1.init_cred(uid=0, gid=0)
-        self.c1 = nfs4client.NFS4Client(self.server, self.port,
-                                        summary=self.summary)
-        self.c1.set_cred(self.cred1)
-        self.c1.null()
-        c = self.c1.new_client("DS.init_%s" % self.server)
-        # This is a hack to ensure MDS/DS communication path is at least
-        # as wide as the client/MDS channel (at least for linux client)
-        fore_attrs = type4.channel_attrs4(0, 16384, 16384, 2868, 8, 8, [])
-        self.sess = c.create_session(fore_attrs=fore_attrs)
-        self.make_root()
+    def reset(self):
+        self.down()
+        self.up()
+
+    def get_netaddr4(self):
+        # STUB server multipathing not supported yet
+        uaddr = '.'.join([self.server,
+                          str(self.port >> 8),
+                          str(self.port & 0xff)])
+        return type4.netaddr4(self.proto, uaddr)
+
+    def get_multipath_netaddr4s(self):
+        netaddr4s = []
+        for addr in self.multipath_servers:
+            server, port = addr
+            uaddr = '.'.join([server,
+                            str(port >> 8),
+                            str(port & 0xff)])
+            proto = "tcp"
+            if server.find(':') >= 0:
+                proto = "tcp6"
+
+            netaddr4s.append(type4.netaddr4(proto, uaddr))
+        return netaddr4s
+
+    def fh_to_name(self, mds_fh):
+        return hashlib.sha1("%r" % mds_fh).hexdigest()
 
     def disconnect(self):
         pass
 
+class DataServer41(DataServer):
     def _execute(self, ops, exceptions=[], delay=5, maxretries=3):
         """ execute the NFS call
         If an error code is specified in the exceptions it means that the
@@ -85,36 +101,28 @@ class DataServer41(object):
                     log.error("Too many retries with DS %s" % self.server)
                     raise Exception("Dataserver communication retry error")
             elif res.status in state_errors:
-                self.disconnect()
-                self.connect()
+                self.reset()
             else:
                 log.error("Unhandled status %s from DS %s" %
                           (nfsstat4[res.status], self.server))
                 raise Exception("Dataserver communication error")
 
-    def get_netaddr4(self):
-        # STUB server multipathing not supported yet
-        uaddr = '.'.join([self.server,
-                          str(self.port >> 8),
-                          str(self.port & 0xff)])
-        return type4.netaddr4(self.proto, uaddr)
-
-    def get_multipath_netaddr4s(self):
-        netaddr4s = []
-        for addr in self.multipath_servers:
-            server, port = addr
-            uaddr = '.'.join([server,
-                            str(port >> 8),
-                            str(port & 0xff)])
-            proto = "tcp"
-            if server.find(':') >= 0:
-                proto = "tcp6"
-
-            netaddr4s.append(type4.netaddr4(proto, uaddr))
-        return netaddr4s
-
+    def connect(self):
+        # only support root with AUTH_SYS for now
+        s1 = rpc.security.instance(rpc.AUTH_SYS)
+        self.cred1 = s1.init_cred(uid=0, gid=0)
+        self.c1 = nfs4client.NFS4Client(self.server, self.port,
+                                        summary=self.summary)
+        self.c1.set_cred(self.cred1)
+        self.c1.null()
+        c = self.c1.new_client("DS.init_%s" % self.server)
+        # This is a hack to ensure MDS/DS communication path is at least
+        # as wide as the client/MDS channel (at least for linux client)
+        fore_attrs = type4.channel_attrs4(0, 16384, 16384, 2868, 8, 8, [])
+        self.sess = c.create_session(fore_attrs=fore_attrs)
 
-    def make_root(self, attrs={const4.FATTR4_MODE:0777}):
+    def make_root(self):
+        attrs = {const4.FATTR4_MODE:0777}
         existing_path = []
         kind = type4.createtype4(const4.NF4DIR)
         for comp in self.path:
@@ -133,12 +141,13 @@ class DataServer41(object):
             raise RuntimeError
         # XXX clean DS directory
 
-    def fh_to_name(self, mds_fh):
-        return hashlib.sha1("%r" % mds_fh).hexdigest()
-
-    def open_file(self, mds_fh, seqid=0,
-                  access=const4.OPEN4_SHARE_ACCESS_BOTH, deny=const4.OPEN4_SHARE_DENY_NONE,
-                  attrs={const4.FATTR4_MODE: 0777}, owner = "mds", mode=const4.GUARDED4):
+    def open_file(self, mds_fh):
+        seqid=0
+        access = const4.OPEN4_SHARE_ACCESS_BOTH
+        deny = const4.OPEN4_SHARE_DENY_NONE
+        attrs = {const4.FATTR4_MODE: 0777}
+        owner = "mds"
+        mode = const4.GUARDED4
         verifier = self.sess.c.verifier
         openflag = type4.openflag4(const4.OPEN4_CREATE, type4.createhow4(mode, attrs, verifier))
         name = self.fh_to_name(mds_fh)
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 14/17] 4.1 server: add support for NFSv3 data servers
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (11 preceding siblings ...)
  2014-06-04 21:02 ` [PATCH pynfs 13/17] dataserver: separate generic and 4.1 code Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
  2014-06-04 21:02 ` [PATCH pynfs 15/17] 4.1 server: get rid of old op_getdeviceinfo Weston Andros Adamson
                   ` (4 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Add the NFSv3 client and a new DataServer class that handles DS ops
using the v3 client.

DataServer3 is not used yet, as it requires flexfile layouts in order
to pass a v3 DS to clients.

Tested with linux client mounting pnfs MDS via v4.1 (disabling pnfs)
and a linux server acting as the v3 DS.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/dataserver.py | 100 ++++++++++++++++++++++++++++
 nfs4.1/nfs3client.py | 180 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 280 insertions(+)
 create mode 100644 nfs4.1/nfs3client.py

diff --git a/nfs4.1/dataserver.py b/nfs4.1/dataserver.py
index 40364d4..0fcd820 100644
--- a/nfs4.1/dataserver.py
+++ b/nfs4.1/dataserver.py
@@ -3,9 +3,12 @@ import nfs4lib
 import xdrdef.nfs4_type as type4
 from xdrdef.nfs4_pack import NFS4Packer
 import xdrdef.nfs4_const as const4
+import xdrdef.nfs3_type as type3
+import xdrdef.nfs3_const as const3
 import time
 import logging
 import nfs4client
+import nfs3client
 import hashlib
 import sys
 import nfs_ops
@@ -14,6 +17,7 @@ import socket
 log = logging.getLogger("Dataserver Manager")
 
 op4 = nfs_ops.NFS4ops()
+op3 = nfs_ops.NFS3ops()
 
 class DataServer(object):
     def __init__(self, server, port, path, flavor=rpc.AUTH_SYS, active=True, mdsds=True, multipath_servers=None, summary=None):
@@ -203,6 +207,102 @@ class DataServer41(DataServer):
         attrdict = res.resarray[-1].obj_attributes
         return attrdict.get(const4.FATTR4_SIZE, 0)
 
+class DataServer3(DataServer):
+    def _execute(self, procnum, procarg, exceptions=(), delay=5, maxretries=3):
+        """ execute the NFS call
+        If an error code is specified in the exceptions it means that the
+        caller wants to handle the error himself
+        """
+        retry_errors = []
+        while True:
+            res = self.c1.proc(procnum, procarg)
+            if res.status == const3.NFS3_OK or res.status in exceptions:
+                return res
+            elif res.status in retry_errors:
+                if maxretries > 0:
+                    maxretries -= 1
+                    time.sleep(delay)
+                else:
+                    log.error("Too many retries with DS %s" % self.server)
+                    raise Exception("Dataserver communication retry error")
+            else:
+                log.error("Unhandled status %s from DS %s" %
+                          (const3.nfsstat3[res.status], self.server))
+                raise Exception("Dataserver communication error")
+
+    def connect(self):
+        # only support root with AUTH_SYS for now
+        s1 = rpc.security.instance(rpc.AUTH_SYS)
+        self.cred1 = s1.init_cred(uid=0, gid=0)
+        self.c1 = nfs3client.NFS3Client(self.server, self.port,
+                                        summary=self.summary)
+        self.c1.set_cred(self.cred1)
+        self.rootfh = type3.nfs_fh3(self.c1.mntclnt.get_rootfh(self.path))
+        self.c1.null()
+
+    def make_root(self):
+        """ don't actually make a root path - we must use it as the export """
+        need = const3.ACCESS3_READ | const3.ACCESS3_LOOKUP | \
+               const3.ACCESS3_MODIFY | const3.ACCESS3_EXTEND
+        arg = op3.access(self.rootfh, need)
+        res = self._execute(const3.NFSPROC3_ACCESS, arg)
+        if res.resok.access != need:
+            raise RuntimeError
+        # XXX clean DS directory
+
+    def open_file(self, mds_fh):
+        name = self.fh_to_name(mds_fh)
+        where = type3.diropargs3(self.rootfh, name)
+        attr = type3.sattr3(mode=type3.set_mode3(True, 0777),
+                            uid=type3.set_uid3(True, 0),
+                            gid=type3.set_gid3(True, 0),
+                            size=type3.set_size3(False),
+                            atime=type3.set_atime(False),
+                            mtime=type3.set_mtime(False))
+        how = type3.createhow3(const3.GUARDED, attr)
+        arg = op3.create(where, how)
+        res = self._execute(const3.NFSPROC3_CREATE, arg,
+                            exceptions=(const3.NFS3ERR_EXIST,))
+
+        if res.status == const3.NFS3_OK:
+            self.filehandles[mds_fh] = (res.resok.obj.handle, None)
+
+        else:
+            arg = op3.lookup(type3.diropargs3(self.rootfh, name))
+            res = self._execute(const3.NFSPROC3_LOOKUP, arg)
+
+            self.filehandles[mds_fh] = (res.resok.object, None)
+
+    def close_file(self, mds_fh):
+        del self.filehandles[mds_fh]
+
+    def read(self, fh, pos, count):
+        arg = op3.read(fh, pos, count)
+        res = self._execute(const3.NFSPROC3_READ, arg)
+        # XXX check res.status?
+        return res.resok.data
+
+    def write(self, fh, pos, data):
+        arg = op3.write(fh, pos, len(data), const3.FILE_SYNC, data)
+        # There are all sorts of error handling issues here
+        res = self._execute(const3.NFSPROC3_WRITE, arg)
+
+    def truncate(self, fh, size):
+        attr = type3.sattr3(mode=type3.set_mode3(False),
+                            uid=type3.set_uid3(False),
+                            gid=type3.set_gid3(False),
+                            size=type3.set_size3(True, size),
+                            atime=type3.set_atime(False),
+                            mtime=type3.set_mtime(False))
+        arg = op3.setattr(fh, attr, type3.sattrguard3(check=False))
+        res = self._execute(const3.NFSPROC3_SETATTR, arg)
+
+    def get_size(self, fh):
+        arg = op3.getattr(fh)
+        res = self._execute(const3.NFSPROC3_GETATTR, arg)
+        # XXX check res.status?
+        return res.resok.obj_attributes.size
+
 
 class DSDevice(object):
     def __init__(self, mdsds):
diff --git a/nfs4.1/nfs3client.py b/nfs4.1/nfs3client.py
new file mode 100644
index 0000000..79a6f0e
--- /dev/null
+++ b/nfs4.1/nfs3client.py
@@ -0,0 +1,180 @@
+import use_local # HACK so don't have to rebuild constantly
+import rpc
+import nfs4lib
+#from nfs4lib import NFS4Error, NFS4Replay, inc_u32
+from xdrdef.sctrl_pack import SCTRLPacker, SCTRLUnpacker
+from xdrdef.nfs3_type import *
+from xdrdef.nfs3_const import *
+from xdrdef.nfs3_pack import NFS3Packer, NFS3Unpacker
+from xdrdef.mnt3_type import *
+from xdrdef.mnt3_const import *
+from xdrdef.mnt3_pack import MNT3Packer, MNT3Unpacker
+from xdrdef.portmap_type import *
+from xdrdef.portmap_const import *
+from xdrdef.portmap_pack import PORTMAPPacker, PORTMAPUnpacker
+import nfs_ops
+import time, struct
+import threading
+import hmac
+import os.path
+
+import traceback
+import logging
+logging.basicConfig(level=logging.INFO,
+                    format="%(levelname)-7s:%(name)s:%(message)s")
+log_cb = logging.getLogger("nfs.client.cb")
+
+op3 = nfs_ops.NFS3ops()
+
+class PORTMAPClient(rpc.Client):
+    def __init__(self, host='localhost', port=PMAP_PORT):
+        rpc.Client.__init__(self, PMAP_PROG, PMAP_VERS)
+        self.server_address = (host, port)
+        self.c1 = self.connect(self.server_address)
+
+    def proc_async(self, procnum, procarg, credinfo=None, pipe=None,
+                   checks=True, packer=PORTMAPPacker):
+        if credinfo is None:
+            credinfo = self.default_cred
+        if pipe is None:
+            pipe = self.c1
+        p = packer(check_enum=checks, check_array=checks)
+        arg_packer = getattr(p, 'pack_%s' % procarg.__class__.__name__)
+        arg_packer(procarg)
+        return self.send_call(pipe, procnum, p.get_buffer(), credinfo)
+
+    def proc(self, procnum, procarg, restypename, **kwargs):
+        xid = self.proc_async(procnum, procarg, **kwargs)
+        pipe = kwargs.get("pipe", None)
+        res = self.listen(xid, restypename, pipe=pipe)
+        return res
+
+    def listen(self, xid, restypename, pipe=None, timeout=10.0):
+        if pipe is None:
+            pipe = self.c1
+        header, data = pipe.listen(xid, timeout)
+        if data:
+            p = PORTMAPUnpacker(data)
+            res_unpacker = getattr(p, 'unpack_%s' % restypename)
+            data = res_unpacker()
+        return data
+
+    def get_port(self, prog, vers):
+        arg = mapping(prog, vers, IPPROTO_TCP, 0)
+
+        res = self.proc(PMAPPROC_GETPORT, arg, 'uint')
+        return res
+
+class Mnt3Client(rpc.Client):
+    def __init__(self, host='localhost', port=None):
+        rpc.Client.__init__(self, MOUNT_PROGRAM, MOUNT_V3)
+        self.server_address = (host, port)
+        self.c1 = self.connect(self.server_address)
+
+    def proc_async(self, procnum, procarg, credinfo=None, pipe=None,
+                   checks=True, packer=MNT3Packer):
+        if credinfo is None:
+            credinfo = self.default_cred
+        if pipe is None:
+            pipe = self.c1
+        p = packer(check_enum=checks, check_array=checks)
+        arg_packer = getattr(p, 'pack_%s' % procarg.__class__.__name__)
+        arg_packer(procarg)
+        return self.send_call(pipe, procnum, p.get_buffer(), credinfo)
+
+    def proc(self, procnum, procarg, restypename, **kwargs):
+        xid = self.proc_async(procnum, procarg, **kwargs)
+        pipe = kwargs.get("pipe", None)
+        res = self.listen(xid, restypename, pipe=pipe)
+        return res
+
+    def listen(self, xid, restypename, pipe=None, timeout=10.0):
+        if pipe is None:
+            pipe = self.c1
+        header, data = pipe.listen(xid, timeout)
+        if data:
+            p = MNT3Unpacker(data)
+            res_unpacker = getattr(p, 'unpack_%s' % restypename)
+            data = res_unpacker()
+        return data
+
+    def get_rootfh(self, export):
+
+        class dirpath(str):
+            pass
+
+        arg = dirpath('/' + os.path.join(*export))
+        res = self.proc(MOUNTPROC3_MNT, arg, 'mountres3')
+        return res.mountinfo.fhandle
+
+class NFS3Client(rpc.Client):
+    def __init__(self, host='localhost', port=None, ctrl_proc=16, summary=None):
+        rpc.Client.__init__(self, 100003, 3)
+        #self.prog = 0x40000000
+        #self.versions = [1] # List of supported versions of prog
+
+        #self.minorversion = minorversion
+        #self.minor_versions = [minorversion]
+        #self.tag = "default tag"
+        #self.impl_id = nfs_impl_id4("citi.umich.edu", "pynfs X.X",
+        #                            nfs4lib.get_nfstime())
+
+        self.portmap = PORTMAPClient(host=host)
+        self.mntport = self.portmap.get_port(MOUNT_PROGRAM, MOUNT_V3)
+        if not port:
+            self.port = self.portmap.get_port(100003, 3)
+        else:
+            self.port = port
+
+        self.verifier = struct.pack('>d', time.time())
+        self.server_address = (host, self.port)
+        self.c1 = self.connect(self.server_address)
+        #self.sessions = {} # XXX Really, this should be per server
+        self.ctrl_proc = ctrl_proc
+        self.summary = summary
+        self.mntclnt = Mnt3Client(host=host, port=self.mntport)
+
+    def set_cred(self, credinfo):
+        self.default_cred = credinfo
+
+    def null_async(self, data=""):
+        return self.send_call(self.c1, 0, data)
+
+    def null(self, *args, **kwargs):
+        xid = self.null_async(*args, **kwargs)
+        return self.listen(xid)
+
+    def proc_async(self, procnum, procarg, credinfo=None, pipe=None,
+                   checks=True, packer=NFS3Packer):
+        if credinfo is None:
+            credinfo = self.default_cred
+        if pipe is None:
+            pipe = self.c1
+        p = packer(check_enum=checks, check_array=checks)
+        arg_packer = getattr(p, 'pack_%s' % procarg.__class__.__name__)
+        arg_packer(procarg)
+        return self.send_call(pipe, procnum, p.get_buffer(), credinfo)
+
+    def proc(self, procnum, procarg, **kwargs):
+        xid = self.proc_async(procnum, procarg, **kwargs)
+        pipe = kwargs.get("pipe", None)
+        res = self.listen(xid, procarg=procarg, pipe=pipe)
+        if self.summary:
+            self.summary.show_op('call v3 %s:%s' % self.server_address,
+                [ procarg.__class__.__name__.lower()[:-1 * len('3args')] ],
+                nfsstat3[res.status])
+        return res
+
+    def listen(self, xid, procarg=None, pipe=None, timeout=10.0):
+        if pipe is None:
+            pipe = self.c1
+        header, data = pipe.listen(xid, timeout)
+        if data:
+            p = NFS3Unpacker(data)
+            argname = procarg.__class__.__name__
+            # FOO3args -> FOO3res
+            resname = argname[:-4] + 'res'
+            res_unpacker = getattr(p, 'unpack_%s' % resname)
+            data = res_unpacker()
+        return data
+
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 15/17] 4.1 server: get rid of old op_getdeviceinfo
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (12 preceding siblings ...)
  2014-06-04 21:02 ` [PATCH pynfs 14/17] 4.1 server: add support for NFSv3 data servers Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
  2014-06-04 21:02 ` [PATCH pynfs 15/17] nfs41 svr: " Weston Andros Adamson
                   ` (3 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

This has been commented out for a while.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/nfs4server.py | 21 ---------------------
 1 file changed, 21 deletions(-)

diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
index 088e8c4..594e9d5 100755
--- a/nfs4.1/nfs4server.py
+++ b/nfs4.1/nfs4server.py
@@ -1870,27 +1870,6 @@ class NFS4Server(rpc.Server):
                                   new_cookie >= len(list))
         return encode_status(NFS4_OK, res)
 
-#     def op_getdeviceinfo(self, arg, env): # STUB
-#         check_session(env)
-#         # check_cfh(env)
-#         # fs = env.cfh.fs
-#         # STUB - only deals with block volumes
-#         kind = arg.gdia_layout_type
-#         if kind != LAYOUT4_BLOCK_VOLUME:
-#             return encode_status(NFS4ERR_INVAL)
-#         # STUB - want to pull this from fs, not block module
-#         d = block.devices.get(arg.gdia_device_id, None)
-#         if d is None:
-#             return encode_status(NFS4ERR_INVAL)
-#         address = device_addr4(LAYOUT4_BLOCK_VOLUME, d.get_addr())
-#         # Check that we don't exceed count
-#         p = nfs4lib.FancyNFS4Packer()
-#         p.pack_device_addr4(address)
-#         if len(p.get_buffer()) > arg.gdia_maxcount:
-#             return encode_status(NFS4ERR_TOOSMALL, gdir_mincount = len(p.get_buffer()))
-#         res = GETDEVICEINFO4resok(address, 0)
-#         return encode_status(NFS4_OK, res)
-
     def op_getdeviceinfo(self, arg, env): # STUB
         # STUB - ignoring notifications
         check_session(env)
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 15/17] nfs41 svr: get rid of old op_getdeviceinfo
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (13 preceding siblings ...)
  2014-06-04 21:02 ` [PATCH pynfs 15/17] 4.1 server: get rid of old op_getdeviceinfo Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
  2014-06-04 21:02 ` [PATCH pynfs 16/17] rpc: on socket error, close and mark pipe inactive Weston Andros Adamson
                   ` (2 subsequent siblings)
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

This has been commented out for a while.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/nfs4server.py | 21 ---------------------
 1 file changed, 21 deletions(-)

diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
index 062e3c4..27e2352 100755
--- a/nfs4.1/nfs4server.py
+++ b/nfs4.1/nfs4server.py
@@ -1870,27 +1870,6 @@ class NFS4Server(rpc.Server):
                                   new_cookie >= len(list))
         return encode_status(NFS4_OK, res)
 
-#     def op_getdeviceinfo(self, arg, env): # STUB
-#         check_session(env)
-#         # check_cfh(env)
-#         # fs = env.cfh.fs
-#         # STUB - only deals with block volumes
-#         kind = arg.gdia_layout_type
-#         if kind != LAYOUT4_BLOCK_VOLUME:
-#             return encode_status(NFS4ERR_INVAL)
-#         # STUB - want to pull this from fs, not block module
-#         d = block.devices.get(arg.gdia_device_id, None)
-#         if d is None:
-#             return encode_status(NFS4ERR_INVAL)
-#         address = device_addr4(LAYOUT4_BLOCK_VOLUME, d.get_addr())
-#         # Check that we don't exceed count
-#         p = nfs4lib.FancyNFS4Packer()
-#         p.pack_device_addr4(address)
-#         if len(p.get_buffer()) > arg.gdia_maxcount:
-#             return encode_status(NFS4ERR_TOOSMALL, gdir_mincount = len(p.get_buffer()))
-#         res = GETDEVICEINFO4resok(address, 0)
-#         return encode_status(NFS4_OK, res)
-
     def op_getdeviceinfo(self, arg, env): # STUB
         # STUB - ignoring notifications
         check_session(env)
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 16/17] rpc: on socket error, close and mark pipe inactive
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (14 preceding siblings ...)
  2014-06-04 21:02 ` [PATCH pynfs 15/17] nfs41 svr: " Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
  2014-06-04 21:02 ` [PATCH pynfs 17/17] nfs3clnt: reconnect when sending on inactive pipe Weston Andros Adamson
       [not found] ` <1401915726-29092-6-git-send-email-dros@primarydata.com>
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

call _event_close() on socket errors instead of tracing back and mark the
pipe as inactive so callers can reconnect as needed.

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 rpc/rpc.py | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/rpc/rpc.py b/rpc/rpc.py
index 250e945..4801883 100644
--- a/rpc/rpc.py
+++ b/rpc/rpc.py
@@ -351,6 +351,7 @@ class RpcPipe(Pipe):
         self._pending = {} # {xid:defer}
         self._lock = threading.Lock() # Protects fields below
         self._xid = 0
+        self.set_active()
 
     def _get_xid(self):
         with self._lock:
@@ -358,6 +359,15 @@ class RpcPipe(Pipe):
             self._xid = inc_u32(out)
         return out
 
+    def set_active(self):
+        self._active = True
+
+    def clear_active(self):
+        self._active = False
+
+    def is_active(self):
+        return self._active
+
     def listen(self, xid, timeout=None):
         """Wait for a reply to a CALL."""
         self._pending[xid].wait(timeout)
@@ -500,15 +510,24 @@ class ConnectionHandler(object):
                 log_p.warn(1, "polling error from %i" % fd)
                 # STUB - now what?
             for fd in w:
-                self._event_write(fd)
+                try:
+                    self._event_write(fd)
+                except socket.error, e:
+                    self._event_close(fd)
             for fd in r:
                 if fd in self.listeners:
-                    self._event_connect_incoming(fd)
+                    try:
+                        self._event_connect_incoming(fd)
+                    except socket.error, e:
+                        self._event_close(fd)
                 elif fd == self._alarm_poll.fileno():
                     commands = self._alarm_poll.recv(self.rsize)
                     for c in commands:
                         data = self._alarm.pop()
-                        switch[c](data)
+                        try:
+                            switch[c](data)
+                        except socket.error, e:
+                            self._event_close(fd)
                 else:
                     try:
                         data = self.sockets[fd].recv_records(self.rsize)
@@ -557,6 +576,7 @@ class ConnectionHandler(object):
         self.writelist -= temp
         self.readlist -= temp
         self.errlist -= temp
+        self.sockets[fd].clear_active()
         self.sockets[fd].close()
         del self.sockets[fd]
 
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH pynfs 17/17] nfs3clnt: reconnect when sending on inactive pipe
  2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
                   ` (15 preceding siblings ...)
  2014-06-04 21:02 ` [PATCH pynfs 16/17] rpc: on socket error, close and mark pipe inactive Weston Andros Adamson
@ 2014-06-04 21:02 ` Weston Andros Adamson
       [not found] ` <1401915726-29092-6-git-send-email-dros@primarydata.com>
  17 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-04 21:02 UTC (permalink / raw)
  To: bfields; +Cc: linux-nfs, Weston Andros Adamson

Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
---
 nfs4.1/nfs3client.py | 45 +++++++++++++++++++++++++--------------------
 1 file changed, 25 insertions(+), 20 deletions(-)

diff --git a/nfs4.1/nfs3client.py b/nfs4.1/nfs3client.py
index 79a6f0e..176765c 100644
--- a/nfs4.1/nfs3client.py
+++ b/nfs4.1/nfs3client.py
@@ -30,14 +30,19 @@ class PORTMAPClient(rpc.Client):
     def __init__(self, host='localhost', port=PMAP_PORT):
         rpc.Client.__init__(self, PMAP_PROG, PMAP_VERS)
         self.server_address = (host, port)
-        self.c1 = self.connect(self.server_address)
+        self._pipe = None
+
+    def get_pipe(self):
+        if not self._pipe or not self._pipe.is_active():
+           self._pipe = self.connect(self.server_address)
+        return self._pipe
 
     def proc_async(self, procnum, procarg, credinfo=None, pipe=None,
                    checks=True, packer=PORTMAPPacker):
         if credinfo is None:
             credinfo = self.default_cred
         if pipe is None:
-            pipe = self.c1
+            pipe = self.get_pipe()
         p = packer(check_enum=checks, check_array=checks)
         arg_packer = getattr(p, 'pack_%s' % procarg.__class__.__name__)
         arg_packer(procarg)
@@ -51,7 +56,7 @@ class PORTMAPClient(rpc.Client):
 
     def listen(self, xid, restypename, pipe=None, timeout=10.0):
         if pipe is None:
-            pipe = self.c1
+            pipe = self.get_pipe()
         header, data = pipe.listen(xid, timeout)
         if data:
             p = PORTMAPUnpacker(data)
@@ -69,14 +74,19 @@ class Mnt3Client(rpc.Client):
     def __init__(self, host='localhost', port=None):
         rpc.Client.__init__(self, MOUNT_PROGRAM, MOUNT_V3)
         self.server_address = (host, port)
-        self.c1 = self.connect(self.server_address)
+        self._pipe = None
+
+    def get_pipe(self):
+        if not self._pipe or not self._pipe.is_active():
+            self._pipe = self.connect(self.server_address)
+        return self._pipe
 
     def proc_async(self, procnum, procarg, credinfo=None, pipe=None,
                    checks=True, packer=MNT3Packer):
         if credinfo is None:
             credinfo = self.default_cred
         if pipe is None:
-            pipe = self.c1
+            pipe = self.get_pipe()
         p = packer(check_enum=checks, check_array=checks)
         arg_packer = getattr(p, 'pack_%s' % procarg.__class__.__name__)
         arg_packer(procarg)
@@ -90,7 +100,7 @@ class Mnt3Client(rpc.Client):
 
     def listen(self, xid, restypename, pipe=None, timeout=10.0):
         if pipe is None:
-            pipe = self.c1
+            pipe = self.get_pipe()
         header, data = pipe.listen(xid, timeout)
         if data:
             p = MNT3Unpacker(data)
@@ -110,15 +120,6 @@ class Mnt3Client(rpc.Client):
 class NFS3Client(rpc.Client):
     def __init__(self, host='localhost', port=None, ctrl_proc=16, summary=None):
         rpc.Client.__init__(self, 100003, 3)
-        #self.prog = 0x40000000
-        #self.versions = [1] # List of supported versions of prog
-
-        #self.minorversion = minorversion
-        #self.minor_versions = [minorversion]
-        #self.tag = "default tag"
-        #self.impl_id = nfs_impl_id4("citi.umich.edu", "pynfs X.X",
-        #                            nfs4lib.get_nfstime())
-
         self.portmap = PORTMAPClient(host=host)
         self.mntport = self.portmap.get_port(MOUNT_PROGRAM, MOUNT_V3)
         if not port:
@@ -128,17 +129,21 @@ class NFS3Client(rpc.Client):
 
         self.verifier = struct.pack('>d', time.time())
         self.server_address = (host, self.port)
-        self.c1 = self.connect(self.server_address)
-        #self.sessions = {} # XXX Really, this should be per server
         self.ctrl_proc = ctrl_proc
         self.summary = summary
+        self._pipe = None
         self.mntclnt = Mnt3Client(host=host, port=self.mntport)
 
+    def get_pipe(self):
+        if not self._pipe or not self._pipe.is_active():
+            self._pipe = self.connect(self.server_address)
+        return self._pipe
+
     def set_cred(self, credinfo):
         self.default_cred = credinfo
 
     def null_async(self, data=""):
-        return self.send_call(self.c1, 0, data)
+        return self.send_call(self.get_pipe(), 0, data)
 
     def null(self, *args, **kwargs):
         xid = self.null_async(*args, **kwargs)
@@ -149,7 +154,7 @@ class NFS3Client(rpc.Client):
         if credinfo is None:
             credinfo = self.default_cred
         if pipe is None:
-            pipe = self.c1
+            pipe = self.get_pipe()
         p = packer(check_enum=checks, check_array=checks)
         arg_packer = getattr(p, 'pack_%s' % procarg.__class__.__name__)
         arg_packer(procarg)
@@ -167,7 +172,7 @@ class NFS3Client(rpc.Client):
 
     def listen(self, xid, procarg=None, pipe=None, timeout=10.0):
         if pipe is None:
-            pipe = self.c1
+            pipe = self.get_pipe()
         header, data = pipe.listen(xid, timeout)
         if data:
             p = NFS3Unpacker(data)
-- 
1.8.5.2 (Apple Git-48)


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session
  2014-06-04 21:01 ` [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session Weston Andros Adamson
@ 2014-06-05  2:26   ` J. Bruce Fields
  2014-06-05 12:54     ` Weston Andros Adamson
  0 siblings, 1 reply; 31+ messages in thread
From: J. Bruce Fields @ 2014-06-05  2:26 UTC (permalink / raw)
  To: Weston Andros Adamson; +Cc: linux-nfs

On Wed, Jun 04, 2014 at 05:01:49PM -0400, Weston Andros Adamson wrote:
> Send RECLAIM_COMPLETE after CREATE_SESSION. This enables backend
> communication to v4.1 linux nfsd servers (i/o through MDS only).

If that's unconditionally adding a reclaim_complete after every
create_sesion, that's probably not what we want since there are tests
that want to do the reclaim_complete by hand.  You probably want to use
new_client_session() instead?  See 219cf7b7f7d0 and the preceding
commit.

--b.

> 
> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
> ---
>  nfs4.1/nfs4client.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
> index e750728..0d94a42 100644
> --- a/nfs4.1/nfs4client.py
> +++ b/nfs4.1/nfs4client.py
> @@ -367,6 +367,7 @@ class ClientRecord(object):
>          self.seqid = inc_u32(csr.csr_sequence) # XXX Do we need to check this?
>          sess = SessionRecord(csr, self)
>          self.c.sessions[sess.sessionid] = sess
> +        sess.compound([op.reclaim_complete(FALSE)])
>          return sess
>  
>      def _cb_hook(self, prefix, opname, funct):
> -- 
> 1.8.5.2 (Apple Git-48)
> 

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-04 21:01 ` [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations Weston Andros Adamson
@ 2014-06-05  2:29   ` J. Bruce Fields
  2014-06-05 12:22     ` Trond Myklebust
  0 siblings, 1 reply; 31+ messages in thread
From: J. Bruce Fields @ 2014-06-05  2:29 UTC (permalink / raw)
  To: Weston Andros Adamson; +Cc: linux-nfs

On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
> Just return ok!

Technically it should record whether or not the reclaim_complete has
happened and return a GRACE error on any non-reclaim open performed
before the reclaim_complete--but for your purposes you may not care...

--b.

> 
> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
> ---
>  nfs4.1/nfs4server.py | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
> index 65fb9af..3607dc0 100755
> --- a/nfs4.1/nfs4server.py
> +++ b/nfs4.1/nfs4server.py
> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
>          with find_state(env, arg.deleg_stateid, allow_0=False) as state:
>              state.delegreturn()
>          return encode_status(NFS4_OK)
> +
> +    def op_reclaim_complete(self, arg, env):
> +        return encode_status(NFS4_OK)
>      
>      def op_getdevicelist(self, arg, env): # STUB
>          check_session(env)
> -- 
> 1.8.5.2 (Apple Git-48)
> 

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 05/17] move .x files to subdir 'xdrdef'
       [not found] ` <1401915726-29092-6-git-send-email-dros@primarydata.com>
@ 2014-06-05  2:31   ` J. Bruce Fields
  2014-06-05 12:51     ` Weston Andros Adamson
  0 siblings, 1 reply; 31+ messages in thread
From: J. Bruce Fields @ 2014-06-05  2:31 UTC (permalink / raw)
  To: Weston Andros Adamson; +Cc: linux-nfs

On Wed, Jun 04, 2014 at 05:01:53PM -0400, Weston Andros Adamson wrote:
>  nfs4.1/nfs4.x                               | 3290 ---------------------------
...
>  nfs4.1/xdrdef/nfs4.x                        | 3290 +++++++++++++++++++++++++++

git should be able to show that as a rename instead of a huge add/delete
pair.  Maybe git-format-patch needs the -M option or something?

--b.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 12/17] add mntv3, portmapv2 and nfsv3 .x files
  2014-06-04 21:02 ` [PATCH pynfs 12/17] add mntv3, portmapv2 and nfsv3 .x files Weston Andros Adamson
@ 2014-06-05  2:34   ` J. Bruce Fields
  0 siblings, 0 replies; 31+ messages in thread
From: J. Bruce Fields @ 2014-06-05  2:34 UTC (permalink / raw)
  To: Weston Andros Adamson; +Cc: linux-nfs

On Wed, Jun 04, 2014 at 05:02:00PM -0400, Weston Andros Adamson wrote:
> mntv3 and nfsv3 .x files made from IETF specs by Tom Haynes
> <thomas.haynes@primarydata.com>.

Some folks actually submitted v3 client code at some point and I never
managed to get it merged.  Maybe I should dig that up again while we're
doing this....  But probably it's still out of scope.

--b.

> 
> portmap2 .x files made from IETF specs by Weston Andros Adamson
> <dros@primarydata.com>.
> 
> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
> ---
>  nfs4.1/xdrdef/mnt3.x    |  68 ++++
>  nfs4.1/xdrdef/nfs3.x    | 818 ++++++++++++++++++++++++++++++++++++++++++++++++
>  nfs4.1/xdrdef/portmap.x |  51 +++
>  3 files changed, 937 insertions(+)
>  create mode 100644 nfs4.1/xdrdef/mnt3.x
>  create mode 100644 nfs4.1/xdrdef/nfs3.x
>  create mode 100644 nfs4.1/xdrdef/portmap.x
> 
> diff --git a/nfs4.1/xdrdef/mnt3.x b/nfs4.1/xdrdef/mnt3.x
> new file mode 100644
> index 0000000..b632322
> --- /dev/null
> +++ b/nfs4.1/xdrdef/mnt3.x
> @@ -0,0 +1,68 @@
> +const MNTPATHLEN = 1024;   /* Maximum bytes in a path name */
> +const MNTNAMLEN = 255;     /* Maximum bytes in a name */
> +const FHSIZE3 = 64;        /* Maximum bytes in a V3 file handle */
> +
> +typedef opaque  fhandle3 <FHSIZE3>;
> +typedef string  dirpath <MNTPATHLEN>;
> +typedef string  name <MNTNAMLEN>;
> +
> +struct mountres3_ok {
> +        fhandle3        fhandle;
> +        int             auth_flavors <>;
> +};
> +
> +union mountres3 switch (mountstat3 fhs_status) {
> +case MNT3_OK:
> +        mountres3_ok mountinfo;
> +default:
> +        void;
> +};
> +
> +struct mountbody {
> +        name            ml_hostname;
> +        dirpath         ml_directory;
> +        mountlist       ml_next;
> +};
> +
> +typedef mountbody *mountlist;
> +
> +
> +struct groupnode {
> +        name            gr_name;
> +        groups          gr_next;
> +};
> +
> +typedef groupnode *groups;
> +
> +
> +struct exportnode {
> +        dirpath         ex_dir;
> +        groups          ex_groups;
> +        exports         ex_next;
> +};
> +
> +typedef exportnode *exports;
> +
> +enum mountstat3 {
> +        MNT3_OK = 0,                     /* no error */
> +        MNT3ERR_PERM = 1,                /* Not owner */
> +        MNT3ERR_NOENT = 2,               /* No such file or directory */
> +        MNT3ERR_IO = 5,                  /* I/O error */
> +        MNT3ERR_ACCES = 13,              /* Permission denied */
> +        MNT3ERR_NOTDIR = 20,             /* Not a directory */
> +        MNT3ERR_INVAL = 22,              /* Invalid argument */
> +        MNT3ERR_NAMETOOLONG = 63,        /* Filename too long */
> +        MNT3ERR_NOTSUPP = 10004,         /* Operation not supported */
> +        MNT3ERR_SERVERFAULT = 10006      /* A failure on the server */
> +};
> +
> +program MOUNT_PROGRAM {
> +        version MOUNT_V3 {
> +                void            MOUNTPROC3_NULL(void) = 0;
> +                mountres3       MOUNTPROC3_MNT(dirpath) = 1;
> +                mountlist       MOUNTPROC3_DUMP(void) = 2;
> +                void            MOUNTPROC3_UMNT(dirpath) = 3;
> +                void            MOUNTPROC3_UMNTALL(void) = 4;
> +                exports         MOUNTPROC3_EXPORT(void) = 5;
> +        } = 3;
> +} = 100005;
> diff --git a/nfs4.1/xdrdef/nfs3.x b/nfs4.1/xdrdef/nfs3.x
> new file mode 100644
> index 0000000..ddd1395
> --- /dev/null
> +++ b/nfs4.1/xdrdef/nfs3.x
> @@ -0,0 +1,818 @@
> +const NFS3_FHSIZE = 64;        /* Maximum bytes in a V3 file handle */
> +
> +const FALSE = 0;
> +const TRUE = 1;
> +
> +const NFS3_COOKIEVERFSIZE = 8;
> +const NFS3_CREATEVERFSIZE = 8;
> +const NFS3_WRITEVERFSIZE = 8;
> +
> +typedef unsigned hyper uint64;
> +typedef hyper   int64;
> +typedef unsigned int uint32;
> +typedef int    int32;
> +typedef string  filename3 <>;
> +typedef string  nfspath3 <>;
> +typedef uint64  fileid3;
> +typedef uint64  cookie3;
> +typedef opaque  cookieverf3[NFS3_COOKIEVERFSIZE];
> +typedef opaque  createverf3[NFS3_CREATEVERFSIZE];
> +typedef opaque  writeverf3[NFS3_WRITEVERFSIZE];
> +typedef uint32  uid3;
> +typedef uint32  gid3;
> +typedef uint64  size3;
> +typedef uint64  offset3;
> +typedef uint32  mode3;
> +typedef uint32  count3;
> +
> +enum nfsstat3 {
> +        NFS3_OK = 0,
> +        NFS3ERR_PERM = 1,
> +        NFS3ERR_NOENT = 2,
> +        NFS3ERR_IO = 5,
> +        NFS3ERR_NXIO = 6,
> +        NFS3ERR_ACCES = 13,
> +        NFS3ERR_EXIST = 17,
> +        NFS3ERR_XDEV = 18,
> +        NFS3ERR_NODEV = 19,
> +        NFS3ERR_NOTDIR = 20,
> +        NFS3ERR_ISDIR = 21,
> +        NFS3ERR_INVAL = 22,
> +        NFS3ERR_FBIG = 27,
> +        NFS3ERR_NOSPC = 28,
> +        NFS3ERR_ROFS = 30,
> +        NFS3ERR_MLINK = 31,
> +        NFS3ERR_NAMETOOLONG = 63,
> +        NFS3ERR_NOTEMPTY = 66,
> +        NFS3ERR_DQUOT = 69,
> +        NFS3ERR_STALE = 70,
> +        NFS3ERR_REMOTE = 71,
> +        NFS3ERR_BADHANDLE = 10001,
> +        NFS3ERR_NOT_SYNC = 10002,
> +        NFS3ERR_BAD_COOKIE = 10003,
> +        NFS3ERR_NOTSUPP = 10004,
> +        NFS3ERR_TOOSMALL = 10005,
> +        NFS3ERR_SERVERFAULT = 10006,
> +        NFS3ERR_BADTYPE = 10007,
> +        NFS3ERR_JUKEBOX = 10008
> +};
> +
> +enum ftype3 {
> +        NF3REG = 1,
> +        NF3DIR = 2,
> +        NF3BLK = 3,
> +        NF3CHR = 4,
> +        NF3LNK = 5,
> +        NF3SOCK = 6,
> +        NF3FIFO = 7
> +};
> +
> +struct specdata3 {
> +        uint32          specdata1;
> +        uint32          specdata2;
> +};
> +
> +struct nfs_fh3 {
> +        opaque          data <NFS3_FHSIZE>;
> +};
> +
> +struct nfstime3 {
> +        uint32          seconds;
> +        uint32          nseconds;
> +};
> +
> +struct fattr3 {
> +        ftype3          type;
> +        mode3           mode;
> +        uint32          nlink;
> +        uid3            uid;
> +        gid3            gid;
> +        size3           size;
> +        size3           used;
> +        specdata3       rdev;
> +        uint64          fsid;
> +        fileid3         fileid;
> +        nfstime3        atime;
> +        nfstime3        mtime;
> +        nfstime3        ctime;
> +};
> +
> +union post_op_attr switch (bool attributes_follow) {
> +case TRUE:
> +        fattr3 attributes;
> +case FALSE:
> +        void;
> +};
> +
> +struct wcc_attr {
> +        size3           size;
> +        nfstime3        mtime;
> +        nfstime3        ctime;
> +};
> +
> +union pre_op_attr switch (bool attributes_follow) {
> +case TRUE:
> +        wcc_attr attributes;
> +case FALSE:
> +        void;
> +};
> +
> +struct wcc_data {
> +        pre_op_attr     before;
> +        post_op_attr    after;
> +};
> +
> +union post_op_fh3 switch (bool handle_follows) {
> +case TRUE:
> +        nfs_fh3 handle;
> +case FALSE:
> +        void;
> +};
> +
> +enum time_how {
> +        DONT_CHANGE = 0,
> +        SET_TO_SERVER_TIME = 1,
> +        SET_TO_CLIENT_TIME = 2
> +};
> +
> +union set_mode3 switch (bool set_it) {
> +case TRUE:
> +        mode3 mode;
> +default:
> +        void;
> +};
> +
> +union set_uid3 switch (bool set_it) {
> +case TRUE:
> +        uid3 uid;
> +default:
> +        void;
> +};
> +
> +union set_gid3 switch (bool set_it) {
> +case TRUE:
> +        gid3 gid;
> +default:
> +        void;
> +};
> +
> +union set_size3 switch (bool set_it) {
> +case TRUE:
> +        size3 size;
> +default:
> +        void;
> +};
> +
> +union set_atime switch (time_how set_it) {
> +case SET_TO_CLIENT_TIME:
> +        nfstime3 atime;
> +default:
> +        void;
> +};
> +
> +union set_mtime switch (time_how set_it) {
> +case SET_TO_CLIENT_TIME:
> +        nfstime3 mtime;
> +default:
> +        void;
> +};
> +
> +struct sattr3 {
> +        set_mode3       mode;
> +        set_uid3        uid;
> +        set_gid3        gid;
> +        set_size3       size;
> +        set_atime       atime;
> +        set_mtime       mtime;
> +};
> +
> +struct diropargs3 {
> +        nfs_fh3         dir;
> +        filename3       name;
> +};
> +
> +struct GETATTR3args {
> +        nfs_fh3         object;
> +};
> +
> +struct GETATTR3resok {
> +        fattr3          obj_attributes;
> +};
> +
> +union GETATTR3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        GETATTR3resok resok;
> +default:
> +        void;
> +};
> +
> +union sattrguard3 switch (bool check) {
> +case TRUE:
> +        nfstime3 obj_ctime;
> +case FALSE:
> +        void;
> +};
> +
> +struct SETATTR3args {
> +        nfs_fh3         object;
> +        sattr3          new_attributes;
> +        sattrguard3     guard;
> +};
> +
> +struct SETATTR3resok {
> +        wcc_data        obj_wcc;
> +};
> +
> +struct SETATTR3resfail {
> +        wcc_data        obj_wcc;
> +};
> +
> +union SETATTR3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        SETATTR3resok resok;
> +default:
> +        SETATTR3resfail resfail;
> +};
> +
> +struct LOOKUP3args {
> +        diropargs3      what;
> +};
> +
> +struct LOOKUP3resok {
> +        nfs_fh3         object;
> +        post_op_attr    obj_attributes;
> +        post_op_attr    dir_attributes;
> +};
> +
> +struct LOOKUP3resfail {
> +        post_op_attr    dir_attributes;
> +};
> +
> +union LOOKUP3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        LOOKUP3resok resok;
> +default:
> +        LOOKUP3resfail resfail;
> +};
> +
> +const           ACCESS3_READ = 0x0001;
> +const           ACCESS3_LOOKUP = 0x0002;
> +const           ACCESS3_MODIFY = 0x0004;
> +const           ACCESS3_EXTEND = 0x0008;
> +const           ACCESS3_DELETE = 0x0010;
> +const           ACCESS3_EXECUTE = 0x0020;
> +
> +struct ACCESS3args {
> +        nfs_fh3         object;
> +        uint32          access;
> +};
> +
> +struct ACCESS3resok {
> +        post_op_attr    obj_attributes;
> +        uint32          access;
> +};
> +
> +struct ACCESS3resfail {
> +        post_op_attr    obj_attributes;
> +};
> +
> +union ACCESS3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        ACCESS3resok resok;
> +default:
> +        ACCESS3resfail resfail;
> +};
> +
> +struct READLINK3args {
> +        nfs_fh3         symlink;
> +};
> +
> +struct READLINK3resok {
> +        post_op_attr    symlink_attributes;
> +        nfspath3        data;
> +};
> +
> +struct READLINK3resfail {
> +        post_op_attr    symlink_attributes;
> +};
> +
> +union READLINK3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        READLINK3resok resok;
> +default:
> +        READLINK3resfail resfail;
> +};
> +
> +struct READ3args {
> +        nfs_fh3         file;
> +        offset3         offset;
> +        count3          count;
> +};
> +
> +struct READ3resok {
> +        post_op_attr    file_attributes;
> +        count3          count;
> +        bool            eof;
> +        opaque          data <>;
> +};
> +
> +struct READ3resfail {
> +        post_op_attr    file_attributes;
> +};
> +
> +union READ3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        READ3resok resok;
> +default:
> +        READ3resfail resfail;
> +};
> +
> +enum stable_how {
> +        UNSTABLE = 0,
> +        DATA_SYNC = 1,
> +        FILE_SYNC = 2
> +};
> +
> +struct WRITE3args {
> +        nfs_fh3         file;
> +        offset3         offset;
> +        count3          count;
> +        stable_how      stable;
> +        opaque          data <>;
> +};
> +
> +struct WRITE3resok {
> +        wcc_data        file_wcc;
> +        count3          count;
> +        stable_how      committed;
> +        writeverf3      verf;
> +};
> +
> +struct WRITE3resfail {
> +        wcc_data        file_wcc;
> +};
> +
> +union WRITE3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        WRITE3resok resok;
> +default:
> +        WRITE3resfail resfail;
> +};
> +
> +enum createmode3 {
> +        UNCHECKED = 0,
> +        GUARDED = 1,
> +        EXCLUSIVE = 2
> +};
> +
> +union createhow3 switch (createmode3 mode) {
> +case UNCHECKED:
> +case GUARDED:
> +        sattr3 obj_attributes;
> +case EXCLUSIVE:
> +        createverf3 verf;
> +};
> +
> +struct CREATE3args {
> +        diropargs3      where;
> +        createhow3      how;
> +};
> +
> +struct CREATE3resok {
> +        post_op_fh3     obj;
> +        post_op_attr    obj_attributes;
> +        wcc_data        dir_wcc;
> +};
> +
> +struct CREATE3resfail {
> +        wcc_data        dir_wcc;
> +};
> +
> +union CREATE3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        CREATE3resok resok;
> +default:
> +        CREATE3resfail resfail;
> +};
> +
> +struct MKDIR3args {
> +        diropargs3      where;
> +        sattr3          attributes;
> +};
> +
> +struct MKDIR3resok {
> +        post_op_fh3     obj;
> +        post_op_attr    obj_attributes;
> +        wcc_data        dir_wcc;
> +};
> +
> +struct MKDIR3resfail {
> +        wcc_data        dir_wcc;
> +};
> +
> +union MKDIR3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        MKDIR3resok resok;
> +default:
> +        MKDIR3resfail resfail;
> +};
> +
> +struct symlinkdata3 {
> +        sattr3          symlink_attributes;
> +        nfspath3        symlink_data;
> +};
> +
> +struct SYMLINK3args {
> +        diropargs3      where;
> +        symlinkdata3    symlink;
> +};
> +
> +struct SYMLINK3resok {
> +        post_op_fh3     obj;
> +        post_op_attr    obj_attributes;
> +        wcc_data        dir_wcc;
> +};
> +
> +struct SYMLINK3resfail {
> +        wcc_data        dir_wcc;
> +};
> +
> +union SYMLINK3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        SYMLINK3resok resok;
> +default:
> +        SYMLINK3resfail resfail;
> +};
> +
> +struct devicedata3 {
> +        sattr3          dev_attributes;
> +        specdata3       spec;
> +};
> +
> +union mknoddata3 switch (ftype3 type) {
> +case NF3CHR:
> +case NF3BLK:
> +        devicedata3 device;
> +case NF3SOCK:
> +case NF3FIFO:
> +        sattr3 pipe_attributes;
> +default:
> +        void;
> +};
> +
> +struct MKNOD3args {
> +        diropargs3      where;
> +        mknoddata3      what;
> +};
> +
> +struct MKNOD3resok {
> +        post_op_fh3     obj;
> +        post_op_attr    obj_attributes;
> +        wcc_data        dir_wcc;
> +};
> +
> +struct MKNOD3resfail {
> +        wcc_data        dir_wcc;
> +};
> +
> +union MKNOD3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        MKNOD3resok resok;
> +default:
> +        MKNOD3resfail resfail;
> +};
> +
> +struct REMOVE3args {
> +        diropargs3      object;
> +};
> +
> +struct REMOVE3resok {
> +        wcc_data        dir_wcc;
> +};
> +
> +struct REMOVE3resfail {
> +        wcc_data        dir_wcc;
> +};
> +
> +union REMOVE3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        REMOVE3resok resok;
> +default:
> +        REMOVE3resfail resfail;
> +};
> +
> +struct RMDIR3args {
> +        diropargs3      object;
> +};
> +
> +struct RMDIR3resok {
> +        wcc_data        dir_wcc;
> +};
> +
> +struct RMDIR3resfail {
> +        wcc_data        dir_wcc;
> +};
> +
> +union RMDIR3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        RMDIR3resok resok;
> +default:
> +        RMDIR3resfail resfail;
> +};
> +
> +struct RENAME3args {
> +        diropargs3      fromfile;
> +        diropargs3      tofile;
> +};
> +
> +struct RENAME3resok {
> +        wcc_data        fromdir_wcc;
> +        wcc_data        todir_wcc;
> +};
> +
> +struct RENAME3resfail {
> +        wcc_data        fromdir_wcc;
> +        wcc_data        todir_wcc;
> +};
> +
> +union RENAME3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        RENAME3resok resok;
> +default:
> +        RENAME3resfail resfail;
> +};
> +
> +struct LINK3args {
> +        nfs_fh3         file;
> +        diropargs3      link;
> +};
> +
> +struct LINK3resok {
> +        post_op_attr    file_attributes;
> +        wcc_data        linkdir_wcc;
> +};
> +
> +struct LINK3resfail {
> +        post_op_attr    file_attributes;
> +        wcc_data        linkdir_wcc;
> +};
> +
> +union LINK3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        LINK3resok resok;
> +default:
> +        LINK3resfail resfail;
> +};
> +
> +struct READDIR3args {
> +        nfs_fh3         dir;
> +        cookie3         cookie;
> +        cookieverf3     cookieverf;
> +        count3          count;
> +};
> +
> +struct entry3 {
> +        fileid3         fileid;
> +        filename3       name;
> +        cookie3         cookie;
> +        entry3         *nextentry;
> +};
> +
> +struct dirlist3 {
> +        entry3         *entries;
> +        bool            eof;
> +};
> +
> +struct READDIR3resok {
> +        post_op_attr    dir_attributes;
> +        cookieverf3     cookieverf;
> +        dirlist3        reply;
> +};
> +
> +struct READDIR3resfail {
> +        post_op_attr    dir_attributes;
> +};
> +
> +union READDIR3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        READDIR3resok resok;
> +default:
> +        READDIR3resfail resfail;
> +};
> +
> +struct READDIRPLUS3args {
> +        nfs_fh3         dir;
> +        cookie3         cookie;
> +        cookieverf3     cookieverf;
> +        count3          dircount;
> +        count3          maxcount;
> +};
> +
> +struct entryplus3 {
> +        fileid3         fileid;
> +        filename3       name;
> +        cookie3         cookie;
> +        post_op_attr    name_attributes;
> +        post_op_fh3     name_handle;
> +        entryplus3     *nextentry;
> +};
> +
> +struct dirlistplus3 {
> +        entryplus3     *entries;
> +        bool            eof;
> +};
> +
> +struct READDIRPLUS3resok {
> +        post_op_attr    dir_attributes;
> +        cookieverf3     cookieverf;
> +        dirlistplus3    reply;
> +};
> +
> +struct READDIRPLUS3resfail {
> +        post_op_attr    dir_attributes;
> +};
> +
> +union READDIRPLUS3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        READDIRPLUS3resok resok;
> +default:
> +        READDIRPLUS3resfail resfail;
> +};
> +
> +struct FSSTAT3args {
> +        nfs_fh3         fsroot;
> +};
> +
> +struct FSSTAT3resok {
> +        post_op_attr    obj_attributes;
> +        size3           tbytes;
> +        size3           fbytes;
> +        size3           abytes;
> +        size3           tfiles;
> +        size3           ffiles;
> +        size3           afiles;
> +        uint32          invarsec;
> +};
> +
> +struct FSSTAT3resfail {
> +        post_op_attr    obj_attributes;
> +};
> +
> +union FSSTAT3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        FSSTAT3resok resok;
> +default:
> +        FSSTAT3resfail resfail;
> +};
> +
> +const           FSF3_LINK = 0x0001;
> +const           FSF3_SYMLINK = 0x0002;
> +const           FSF3_HOMOGENEOUS = 0x0008;
> +const           FSF3_CANSETTIME = 0x0010;
> +
> +struct FSINFOargs {
> +        nfs_fh3         fsroot;
> +};
> +
> +struct FSINFO3resok {
> +        post_op_attr    obj_attributes;
> +        uint32          rtmax;
> +        uint32          rtpref;
> +        uint32          rtmult;
> +        uint32          wtmax;
> +        uint32          wtpref;
> +        uint32          wtmult;
> +        uint32          dtpref;
> +        size3           maxfilesize;
> +        nfstime3        time_delta;
> +        uint32          properties;
> +};
> +
> +struct FSINFO3resfail {
> +        post_op_attr    obj_attributes;
> +};
> +
> +union FSINFO3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        FSINFO3resok resok;
> +default:
> +        FSINFO3resfail resfail;
> +};
> +
> +struct PATHCONF3args {
> +        nfs_fh3         object;
> +};
> +
> +struct PATHCONF3resok {
> +        post_op_attr    obj_attributes;
> +        uint32          linkmax;
> +        uint32          name_max;
> +        bool            no_trunc;
> +        bool            chown_restricted;
> +        bool            case_insensitive;
> +        bool            case_preserving;
> +};
> +
> +struct PATHCONF3resfail {
> +        post_op_attr    obj_attributes;
> +};
> +
> +union PATHCONF3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        PATHCONF3resok resok;
> +default:
> +        PATHCONF3resfail resfail;
> +};
> +
> +struct COMMIT3args {
> +        nfs_fh3         file;
> +        offset3         offset;
> +        count3          count;
> +};
> +
> +struct COMMIT3resok {
> +        wcc_data        file_wcc;
> +        writeverf3      verf;
> +};
> +
> +struct COMMIT3resfail {
> +        wcc_data        file_wcc;
> +};
> +
> +union COMMIT3res switch (nfsstat3 status) {
> +case NFS3_OK:
> +        COMMIT3resok resok;
> +default:
> +        COMMIT3resfail resfail;
> +};
> +
> +program         NFS_PROGRAM {
> +        version         NFS_V3 {
> +
> +                void
> +                NFSPROC3_NULL(void) = 0;
> +
> +                GETATTR3res
> +                NFSPROC3_GETATTR(GETATTR3args) = 1;
> +
> +                SETATTR3res
> +                NFSPROC3_SETATTR(SETATTR3args) = 2;
> +
> +                LOOKUP3res
> +                NFSPROC3_LOOKUP(LOOKUP3args) = 3;
> +
> +                ACCESS3res
> +                NFSPROC3_ACCESS(ACCESS3args) = 4;
> +
> +                READLINK3res
> +                NFSPROC3_READLINK(READLINK3args) = 5;
> +
> +                READ3res
> +                NFSPROC3_READ(READ3args) = 6;
> +
> +                WRITE3res
> +                NFSPROC3_WRITE(WRITE3args) = 7;
> +
> +                CREATE3res
> +                NFSPROC3_CREATE(CREATE3args) = 8;
> +
> +                MKDIR3res
> +                NFSPROC3_MKDIR(MKDIR3args) = 9;
> +
> +                SYMLINK3res
> +                NFSPROC3_SYMLINK(SYMLINK3args) = 10;
> +
> +                MKNOD3res
> +                NFSPROC3_MKNOD(MKNOD3args) = 11;
> +
> +                REMOVE3res
> +                NFSPROC3_REMOVE(REMOVE3args) = 12;
> +
> +                RMDIR3res
> +                NFSPROC3_RMDIR(RMDIR3args) = 13;
> +
> +                RENAME3res
> +                NFSPROC3_RENAME(RENAME3args) = 14;
> +
> +                LINK3res
> +                NFSPROC3_LINK(LINK3args) = 15;
> +
> +                READDIR3res
> +                NFSPROC3_READDIR(READDIR3args) = 16;
> +
> +                READDIRPLUS3res
> +                NFSPROC3_READDIRPLUS(READDIRPLUS3args) = 17;
> +
> +                FSSTAT3res
> +                NFSPROC3_FSSTAT(FSSTAT3args) = 18;
> +
> +                FSINFO3res
> +                NFSPROC3_FSINFO(FSINFO3args) = 19;
> +
> +                PATHCONF3res
> +                NFSPROC3_PATHCONF(PATHCONF3args) = 20;
> +
> +                COMMIT3res
> +                NFSPROC3_COMMIT(COMMIT3args) = 21;
> +        } = 3;
> +} = 100003;
> diff --git a/nfs4.1/xdrdef/portmap.x b/nfs4.1/xdrdef/portmap.x
> new file mode 100644
> index 0000000..1158d4e
> --- /dev/null
> +++ b/nfs4.1/xdrdef/portmap.x
> @@ -0,0 +1,51 @@
> +const PMAP_PORT = 111;      /* portmapper port number */
> +
> +struct mapping {
> +	unsigned int prog;
> +	unsigned int vers;
> +	unsigned int prot;
> +	unsigned int port;
> +};
> +
> +const IPPROTO_TCP = 6;      /* protocol number for TCP/IP */
> +const IPPROTO_UDP = 17;     /* protocol number for UDP/IP */
> +
> +struct pmaplist {
> +	mapping map;
> +	pmaplist next;
> +};
> +
> +struct call_args {
> +	unsigned int prog;
> +	unsigned int vers;
> +	unsigned int proc;
> +	opaque args<>;
> +};
> +
> +struct call_result {
> +	unsigned int port;
> +	opaque res<>;
> +};
> +
> +program PMAP_PROG {
> +	version PMAP_VERS {
> +		void
> +		PMAPPROC_NULL(void)         = 0;
> +
> +		bool
> +		PMAPPROC_SET(mapping)       = 1;
> +
> +		bool
> +		PMAPPROC_UNSET(mapping)     = 2;
> +
> +		unsigned int
> +		PMAPPROC_GETPORT(mapping)   = 3;
> +
> +		pmaplist
> +		PMAPPROC_DUMP(void)         = 4;
> +
> +		call_result
> +		PMAPPROC_CALLIT(call_args)  = 5;
> +	} = 2;
> +} = 100000;
> +
> -- 
> 1.8.5.2 (Apple Git-48)
> 

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05  2:29   ` J. Bruce Fields
@ 2014-06-05 12:22     ` Trond Myklebust
  2014-06-05 12:58       ` Weston Andros Adamson
  0 siblings, 1 reply; 31+ messages in thread
From: Trond Myklebust @ 2014-06-05 12:22 UTC (permalink / raw)
  To: J. Bruce Fields; +Cc: Weston Andros Adamson, linux-nfs

On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
>> Just return ok!
>
> Technically it should record whether or not the reclaim_complete has
> happened and return a GRACE error on any non-reclaim open performed
> before the reclaim_complete--but for your purposes you may not care...
>

...and a NOGRACE error on any reclaim opens performed by that client
after the reclaim_complete?

> --b.
>
>>
>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
>> ---
>>  nfs4.1/nfs4server.py | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
>> index 65fb9af..3607dc0 100755
>> --- a/nfs4.1/nfs4server.py
>> +++ b/nfs4.1/nfs4server.py
>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
>>          with find_state(env, arg.deleg_stateid, allow_0=False) as state:
>>              state.delegreturn()
>>          return encode_status(NFS4_OK)
>> +
>> +    def op_reclaim_complete(self, arg, env):
>> +        return encode_status(NFS4_OK)
>>
>>      def op_getdevicelist(self, arg, env): # STUB
>>          check_session(env)
>> --
>> 1.8.5.2 (Apple Git-48)
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 05/17] move .x files to subdir 'xdrdef'
  2014-06-05  2:31   ` [PATCH pynfs 05/17] move .x files to subdir 'xdrdef' J. Bruce Fields
@ 2014-06-05 12:51     ` Weston Andros Adamson
  0 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-05 12:51 UTC (permalink / raw)
  To: J. Bruce Fields; +Cc: linux-nfs list

Huh, I didn’t know there was such an option. I’ll repost.

-dros



On Jun 4, 2014, at 10:31 PM, J. Bruce Fields <bfields@fieldses.org> wrote:

> On Wed, Jun 04, 2014 at 05:01:53PM -0400, Weston Andros Adamson wrote:
>> nfs4.1/nfs4.x                               | 3290 ---------------------------
> ...
>> nfs4.1/xdrdef/nfs4.x                        | 3290 +++++++++++++++++++++++++++
> 
> git should be able to show that as a rename instead of a huge add/delete
> pair.  Maybe git-format-patch needs the -M option or something?
> 
> --b.


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session
  2014-06-05  2:26   ` J. Bruce Fields
@ 2014-06-05 12:54     ` Weston Andros Adamson
  0 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-05 12:54 UTC (permalink / raw)
  To: J. Bruce Fields; +Cc: linux-nfs list

On Jun 4, 2014, at 10:26 PM, J. Bruce Fields <bfields@fieldses.org> wrote:

> On Wed, Jun 04, 2014 at 05:01:49PM -0400, Weston Andros Adamson wrote:
>> Send RECLAIM_COMPLETE after CREATE_SESSION. This enables backend
>> communication to v4.1 linux nfsd servers (i/o through MDS only).
> 
> If that's unconditionally adding a reclaim_complete after every
> create_sesion, that's probably not what we want since there are tests
> that want to do the reclaim_complete by hand.  You probably want to use
> new_client_session() instead?  See 219cf7b7f7d0 and the preceding
> commit.
> 
> —b.

Ok, thats a good point, but AFAIC new_client_session is never called - at least by
nfs4server.

I’ll figure out a better way to do this.

-dros

> 
>> 
>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
>> ---
>> nfs4.1/nfs4client.py | 1 +
>> 1 file changed, 1 insertion(+)
>> 
>> diff --git a/nfs4.1/nfs4client.py b/nfs4.1/nfs4client.py
>> index e750728..0d94a42 100644
>> --- a/nfs4.1/nfs4client.py
>> +++ b/nfs4.1/nfs4client.py
>> @@ -367,6 +367,7 @@ class ClientRecord(object):
>>         self.seqid = inc_u32(csr.csr_sequence) # XXX Do we need to check this?
>>         sess = SessionRecord(csr, self)
>>         self.c.sessions[sess.sessionid] = sess
>> +        sess.compound([op.reclaim_complete(FALSE)])
>>         return sess
>> 
>>     def _cb_hook(self, prefix, opname, funct):
>> -- 
>> 1.8.5.2 (Apple Git-48)
>> 


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05 12:22     ` Trond Myklebust
@ 2014-06-05 12:58       ` Weston Andros Adamson
  2014-06-05 13:06         ` J. Bruce Fields
  0 siblings, 1 reply; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-05 12:58 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: J. Bruce Fields, linux-nfs list

Are you saying that the pynfs server supports state recovery? This has not
been my experience. I’ll double check.

-dros



On Jun 5, 2014, at 8:22 AM, Trond Myklebust <trondmy@gmail.com> wrote:

> On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
>>> Just return ok!
>> 
>> Technically it should record whether or not the reclaim_complete has
>> happened and return a GRACE error on any non-reclaim open performed
>> before the reclaim_complete--but for your purposes you may not care...
>> 
> 
> ...and a NOGRACE error on any reclaim opens performed by that client
> after the reclaim_complete?
> 
>> --b.
>> 
>>> 
>>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
>>> ---
>>> nfs4.1/nfs4server.py | 3 +++
>>> 1 file changed, 3 insertions(+)
>>> 
>>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
>>> index 65fb9af..3607dc0 100755
>>> --- a/nfs4.1/nfs4server.py
>>> +++ b/nfs4.1/nfs4server.py
>>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
>>>         with find_state(env, arg.deleg_stateid, allow_0=False) as state:
>>>             state.delegreturn()
>>>         return encode_status(NFS4_OK)
>>> +
>>> +    def op_reclaim_complete(self, arg, env):
>>> +        return encode_status(NFS4_OK)
>>> 
>>>     def op_getdevicelist(self, arg, env): # STUB
>>>         check_session(env)
>>> --
>>> 1.8.5.2 (Apple Git-48)
>>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05 12:58       ` Weston Andros Adamson
@ 2014-06-05 13:06         ` J. Bruce Fields
  2014-06-05 13:18           ` Weston Andros Adamson
  0 siblings, 1 reply; 31+ messages in thread
From: J. Bruce Fields @ 2014-06-05 13:06 UTC (permalink / raw)
  To: Weston Andros Adamson; +Cc: Trond Myklebust, linux-nfs list

On Thu, Jun 05, 2014 at 08:58:01AM -0400, Weston Andros Adamson wrote:
> Are you saying that the pynfs server supports state recovery? This has not
> been my experience. I’ll double check.

If you don't support state recovery, then I think the minimal correct
behavior would be to have no grace period at all: return NO_GRACE on
*every* reclaim operation and GRACE only on non-reclaims not preceded by
a global (one_fs == FALSE) RECLAIM_COMPLETE for that client.

All this does is catch misbehaving clients, and maybe that's not a
priority.  But it's easy enough to implement.

--b.

> -dros
> 
> 
> 
> On Jun 5, 2014, at 8:22 AM, Trond Myklebust <trondmy@gmail.com> wrote:
> 
> > On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> >> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
> >>> Just return ok!
> >> 
> >> Technically it should record whether or not the reclaim_complete has
> >> happened and return a GRACE error on any non-reclaim open performed
> >> before the reclaim_complete--but for your purposes you may not care...
> >> 
> > 
> > ...and a NOGRACE error on any reclaim opens performed by that client
> > after the reclaim_complete?
> > 
> >> --b.
> >> 
> >>> 
> >>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
> >>> ---
> >>> nfs4.1/nfs4server.py | 3 +++
> >>> 1 file changed, 3 insertions(+)
> >>> 
> >>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
> >>> index 65fb9af..3607dc0 100755
> >>> --- a/nfs4.1/nfs4server.py
> >>> +++ b/nfs4.1/nfs4server.py
> >>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
> >>>         with find_state(env, arg.deleg_stateid, allow_0=False) as state:
> >>>             state.delegreturn()
> >>>         return encode_status(NFS4_OK)
> >>> +
> >>> +    def op_reclaim_complete(self, arg, env):
> >>> +        return encode_status(NFS4_OK)
> >>> 
> >>>     def op_getdevicelist(self, arg, env): # STUB
> >>>         check_session(env)
> >>> --
> >>> 1.8.5.2 (Apple Git-48)
> >>> 
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05 13:06         ` J. Bruce Fields
@ 2014-06-05 13:18           ` Weston Andros Adamson
  2014-06-05 13:34             ` Weston Andros Adamson
  0 siblings, 1 reply; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-05 13:18 UTC (permalink / raw)
  To: J. Bruce Fields; +Cc: Trond Myklebust, linux-nfs list


On Jun 5, 2014, at 9:06 AM, J. Bruce Fields <bfields@fieldses.org> wrote:

> On Thu, Jun 05, 2014 at 08:58:01AM -0400, Weston Andros Adamson wrote:
>> Are you saying that the pynfs server supports state recovery? This has not
>> been my experience. I’ll double check.
> 
> If you don't support state recovery, then I think the minimal correct
> behavior would be to have no grace period at all: return NO_GRACE on
> *every* reclaim operation and GRACE only on non-reclaims not preceded by
> a global (one_fs == FALSE) RECLAIM_COMPLETE for that client.
> 
> All this does is catch misbehaving clients, and maybe that's not a
> priority.  But it's easy enough to implement.

Yeah, that sounds good.

-dros

> 
> —b.
> 
>> -dros
>> 
>> 
>> 
>> On Jun 5, 2014, at 8:22 AM, Trond Myklebust <trondmy@gmail.com> wrote:
>> 
>>> On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
>>>>> Just return ok!
>>>> 
>>>> Technically it should record whether or not the reclaim_complete has
>>>> happened and return a GRACE error on any non-reclaim open performed
>>>> before the reclaim_complete--but for your purposes you may not care...
>>>> 
>>> 
>>> ...and a NOGRACE error on any reclaim opens performed by that client
>>> after the reclaim_complete?
>>> 
>>>> --b.
>>>> 
>>>>> 
>>>>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
>>>>> ---
>>>>> nfs4.1/nfs4server.py | 3 +++
>>>>> 1 file changed, 3 insertions(+)
>>>>> 
>>>>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
>>>>> index 65fb9af..3607dc0 100755
>>>>> --- a/nfs4.1/nfs4server.py
>>>>> +++ b/nfs4.1/nfs4server.py
>>>>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
>>>>>        with find_state(env, arg.deleg_stateid, allow_0=False) as state:
>>>>>            state.delegreturn()
>>>>>        return encode_status(NFS4_OK)
>>>>> +
>>>>> +    def op_reclaim_complete(self, arg, env):
>>>>> +        return encode_status(NFS4_OK)
>>>>> 
>>>>>    def op_getdevicelist(self, arg, env): # STUB
>>>>>        check_session(env)
>>>>> --
>>>>> 1.8.5.2 (Apple Git-48)
>>>>> 
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05 13:18           ` Weston Andros Adamson
@ 2014-06-05 13:34             ` Weston Andros Adamson
  2014-06-05 13:41               ` J. Bruce Fields
  0 siblings, 1 reply; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-05 13:34 UTC (permalink / raw)
  To: J. Bruce Fields; +Cc: Trond Myklebust, linux-nfs list

On second thought, I’m just going to drop this patch.

I only added it to avoid a NFS4ERR_NOTSUPP when connecting the file layout
MDS to pynfs DSes, but it is harmless and outside the scope of what I’m doing.

Not worth that can of worms.

-dros



On Jun 5, 2014, at 9:18 AM, Weston Andros Adamson <dros@primarydata.com> wrote:

> 
> On Jun 5, 2014, at 9:06 AM, J. Bruce Fields <bfields@fieldses.org> wrote:
> 
>> On Thu, Jun 05, 2014 at 08:58:01AM -0400, Weston Andros Adamson wrote:
>>> Are you saying that the pynfs server supports state recovery? This has not
>>> been my experience. I’ll double check.
>> 
>> If you don't support state recovery, then I think the minimal correct
>> behavior would be to have no grace period at all: return NO_GRACE on
>> *every* reclaim operation and GRACE only on non-reclaims not preceded by
>> a global (one_fs == FALSE) RECLAIM_COMPLETE for that client.
>> 
>> All this does is catch misbehaving clients, and maybe that's not a
>> priority.  But it's easy enough to implement.
> 
> Yeah, that sounds good.
> 
> -dros
> 
>> 
>> —b.
>> 
>>> -dros
>>> 
>>> 
>>> 
>>> On Jun 5, 2014, at 8:22 AM, Trond Myklebust <trondmy@gmail.com> wrote:
>>> 
>>>> On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>>> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
>>>>>> Just return ok!
>>>>> 
>>>>> Technically it should record whether or not the reclaim_complete has
>>>>> happened and return a GRACE error on any non-reclaim open performed
>>>>> before the reclaim_complete--but for your purposes you may not care...
>>>>> 
>>>> 
>>>> ...and a NOGRACE error on any reclaim opens performed by that client
>>>> after the reclaim_complete?
>>>> 
>>>>> --b.
>>>>> 
>>>>>> 
>>>>>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
>>>>>> ---
>>>>>> nfs4.1/nfs4server.py | 3 +++
>>>>>> 1 file changed, 3 insertions(+)
>>>>>> 
>>>>>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
>>>>>> index 65fb9af..3607dc0 100755
>>>>>> --- a/nfs4.1/nfs4server.py
>>>>>> +++ b/nfs4.1/nfs4server.py
>>>>>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
>>>>>>       with find_state(env, arg.deleg_stateid, allow_0=False) as state:
>>>>>>           state.delegreturn()
>>>>>>       return encode_status(NFS4_OK)
>>>>>> +
>>>>>> +    def op_reclaim_complete(self, arg, env):
>>>>>> +        return encode_status(NFS4_OK)
>>>>>> 
>>>>>>   def op_getdevicelist(self, arg, env): # STUB
>>>>>>       check_session(env)
>>>>>> --
>>>>>> 1.8.5.2 (Apple Git-48)
>>>>>> 
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> 
> 


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05 13:34             ` Weston Andros Adamson
@ 2014-06-05 13:41               ` J. Bruce Fields
  2014-06-05 13:49                 ` Weston Andros Adamson
  0 siblings, 1 reply; 31+ messages in thread
From: J. Bruce Fields @ 2014-06-05 13:41 UTC (permalink / raw)
  To: Weston Andros Adamson; +Cc: Trond Myklebust, linux-nfs list

On Thu, Jun 05, 2014 at 09:34:14AM -0400, Weston Andros Adamson wrote:
> On second thought, I’m just going to drop this patch.
> 
> I only added it to avoid a NFS4ERR_NOTSUPP when connecting the file layout
> MDS to pynfs DSes, but it is harmless and outside the scope of what I’m doing.
> 
> Not worth that can of worms.

The MDS shouldn't really have to handle NOTSUPP on RECLAIM_COMPLETE
(even if yours currently happens to).  I think your no-op patch as it is
would be better than nothing.

--b.

> 
> -dros
> 
> 
> 
> On Jun 5, 2014, at 9:18 AM, Weston Andros Adamson <dros@primarydata.com> wrote:
> 
> > 
> > On Jun 5, 2014, at 9:06 AM, J. Bruce Fields <bfields@fieldses.org> wrote:
> > 
> >> On Thu, Jun 05, 2014 at 08:58:01AM -0400, Weston Andros Adamson wrote:
> >>> Are you saying that the pynfs server supports state recovery? This has not
> >>> been my experience. I’ll double check.
> >> 
> >> If you don't support state recovery, then I think the minimal correct
> >> behavior would be to have no grace period at all: return NO_GRACE on
> >> *every* reclaim operation and GRACE only on non-reclaims not preceded by
> >> a global (one_fs == FALSE) RECLAIM_COMPLETE for that client.
> >> 
> >> All this does is catch misbehaving clients, and maybe that's not a
> >> priority.  But it's easy enough to implement.
> > 
> > Yeah, that sounds good.
> > 
> > -dros
> > 
> >> 
> >> —b.
> >> 
> >>> -dros
> >>> 
> >>> 
> >>> 
> >>> On Jun 5, 2014, at 8:22 AM, Trond Myklebust <trondmy@gmail.com> wrote:
> >>> 
> >>>> On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> >>>>> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
> >>>>>> Just return ok!
> >>>>> 
> >>>>> Technically it should record whether or not the reclaim_complete has
> >>>>> happened and return a GRACE error on any non-reclaim open performed
> >>>>> before the reclaim_complete--but for your purposes you may not care...
> >>>>> 
> >>>> 
> >>>> ...and a NOGRACE error on any reclaim opens performed by that client
> >>>> after the reclaim_complete?
> >>>> 
> >>>>> --b.
> >>>>> 
> >>>>>> 
> >>>>>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
> >>>>>> ---
> >>>>>> nfs4.1/nfs4server.py | 3 +++
> >>>>>> 1 file changed, 3 insertions(+)
> >>>>>> 
> >>>>>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
> >>>>>> index 65fb9af..3607dc0 100755
> >>>>>> --- a/nfs4.1/nfs4server.py
> >>>>>> +++ b/nfs4.1/nfs4server.py
> >>>>>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
> >>>>>>       with find_state(env, arg.deleg_stateid, allow_0=False) as state:
> >>>>>>           state.delegreturn()
> >>>>>>       return encode_status(NFS4_OK)
> >>>>>> +
> >>>>>> +    def op_reclaim_complete(self, arg, env):
> >>>>>> +        return encode_status(NFS4_OK)
> >>>>>> 
> >>>>>>   def op_getdevicelist(self, arg, env): # STUB
> >>>>>>       check_session(env)
> >>>>>> --
> >>>>>> 1.8.5.2 (Apple Git-48)
> >>>>>> 
> >>>>> --
> >>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> >>>>> the body of a message to majordomo@vger.kernel.org
> >>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>> 
> > 
> 

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations
  2014-06-05 13:41               ` J. Bruce Fields
@ 2014-06-05 13:49                 ` Weston Andros Adamson
  0 siblings, 0 replies; 31+ messages in thread
From: Weston Andros Adamson @ 2014-06-05 13:49 UTC (permalink / raw)
  To: J. Bruce Fields; +Cc: Trond Myklebust, linux-nfs list

Well, it works fine without it - the pynfs MDS just ignores the NOTSUPP!

If you really want it I can submit it later, but it’s unneeded and unrelated to
the goals of this patchset.

-dros



On Jun 5, 2014, at 9:41 AM, J. Bruce Fields <bfields@fieldses.org> wrote:

> On Thu, Jun 05, 2014 at 09:34:14AM -0400, Weston Andros Adamson wrote:
>> On second thought, I’m just going to drop this patch.
>> 
>> I only added it to avoid a NFS4ERR_NOTSUPP when connecting the file layout
>> MDS to pynfs DSes, but it is harmless and outside the scope of what I’m doing.
>> 
>> Not worth that can of worms.
> 
> The MDS shouldn't really have to handle NOTSUPP on RECLAIM_COMPLETE
> (even if yours currently happens to).  I think your no-op patch as it is
> would be better than nothing.
> 
> --b.
> 
>> 
>> -dros
>> 
>> 
>> 
>> On Jun 5, 2014, at 9:18 AM, Weston Andros Adamson <dros@primarydata.com> wrote:
>> 
>>> 
>>> On Jun 5, 2014, at 9:06 AM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>> 
>>>> On Thu, Jun 05, 2014 at 08:58:01AM -0400, Weston Andros Adamson wrote:
>>>>> Are you saying that the pynfs server supports state recovery? This has not
>>>>> been my experience. I’ll double check.
>>>> 
>>>> If you don't support state recovery, then I think the minimal correct
>>>> behavior would be to have no grace period at all: return NO_GRACE on
>>>> *every* reclaim operation and GRACE only on non-reclaims not preceded by
>>>> a global (one_fs == FALSE) RECLAIM_COMPLETE for that client.
>>>> 
>>>> All this does is catch misbehaving clients, and maybe that's not a
>>>> priority.  But it's easy enough to implement.
>>> 
>>> Yeah, that sounds good.
>>> 
>>> -dros
>>> 
>>>> 
>>>> —b.
>>>> 
>>>>> -dros
>>>>> 
>>>>> 
>>>>> 
>>>>> On Jun 5, 2014, at 8:22 AM, Trond Myklebust <trondmy@gmail.com> wrote:
>>>>> 
>>>>>> On Wed, Jun 4, 2014 at 10:29 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>>>>> On Wed, Jun 04, 2014 at 05:01:50PM -0400, Weston Andros Adamson wrote:
>>>>>>>> Just return ok!
>>>>>>> 
>>>>>>> Technically it should record whether or not the reclaim_complete has
>>>>>>> happened and return a GRACE error on any non-reclaim open performed
>>>>>>> before the reclaim_complete--but for your purposes you may not care...
>>>>>>> 
>>>>>> 
>>>>>> ...and a NOGRACE error on any reclaim opens performed by that client
>>>>>> after the reclaim_complete?
>>>>>> 
>>>>>>> --b.
>>>>>>> 
>>>>>>>> 
>>>>>>>> Signed-off-by: Weston Andros Adamson <dros@primarydata.com>
>>>>>>>> ---
>>>>>>>> nfs4.1/nfs4server.py | 3 +++
>>>>>>>> 1 file changed, 3 insertions(+)
>>>>>>>> 
>>>>>>>> diff --git a/nfs4.1/nfs4server.py b/nfs4.1/nfs4server.py
>>>>>>>> index 65fb9af..3607dc0 100755
>>>>>>>> --- a/nfs4.1/nfs4server.py
>>>>>>>> +++ b/nfs4.1/nfs4server.py
>>>>>>>> @@ -1809,6 +1809,9 @@ class NFS4Server(rpc.Server):
>>>>>>>>      with find_state(env, arg.deleg_stateid, allow_0=False) as state:
>>>>>>>>          state.delegreturn()
>>>>>>>>      return encode_status(NFS4_OK)
>>>>>>>> +
>>>>>>>> +    def op_reclaim_complete(self, arg, env):
>>>>>>>> +        return encode_status(NFS4_OK)
>>>>>>>> 
>>>>>>>>  def op_getdevicelist(self, arg, env): # STUB
>>>>>>>>      check_session(env)
>>>>>>>> --
>>>>>>>> 1.8.5.2 (Apple Git-48)
>>>>>>>> 
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>> 
>>> 
>> 


^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2014-06-05 13:49 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-06-04 21:01 [PATCH pynfs 00/17] prep for flex file layout server Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 01/17] 4.1 client: reclaim_complete after create_session Weston Andros Adamson
2014-06-05  2:26   ` J. Bruce Fields
2014-06-05 12:54     ` Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 02/17] 4.1 server: service RECLAIM_COMPLETE operations Weston Andros Adamson
2014-06-05  2:29   ` J. Bruce Fields
2014-06-05 12:22     ` Trond Myklebust
2014-06-05 12:58       ` Weston Andros Adamson
2014-06-05 13:06         ` J. Bruce Fields
2014-06-05 13:18           ` Weston Andros Adamson
2014-06-05 13:34             ` Weston Andros Adamson
2014-06-05 13:41               ` J. Bruce Fields
2014-06-05 13:49                 ` Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 03/17] dataserver: only catch connection error Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 04/17] 4.1 server: avoid traceback in DS disconnect() Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 06/17] 4.1 client: remove unused imports Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 07/17] 4.1 server: add -v flag & silence random output Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 08/17] 4.1 server: add -s option to print summary of ops Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 09/17] dataserver: make generic interface to ops Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 10/17] dataserver: don't import * from nfs4 specific mods Weston Andros Adamson
2014-06-04 21:01 ` [PATCH pynfs 11/17] 4.1 server: move nfs4_ops.py to nfs_ops.py Weston Andros Adamson
2014-06-04 21:02 ` [PATCH pynfs 12/17] add mntv3, portmapv2 and nfsv3 .x files Weston Andros Adamson
2014-06-05  2:34   ` J. Bruce Fields
2014-06-04 21:02 ` [PATCH pynfs 13/17] dataserver: separate generic and 4.1 code Weston Andros Adamson
2014-06-04 21:02 ` [PATCH pynfs 14/17] 4.1 server: add support for NFSv3 data servers Weston Andros Adamson
2014-06-04 21:02 ` [PATCH pynfs 15/17] 4.1 server: get rid of old op_getdeviceinfo Weston Andros Adamson
2014-06-04 21:02 ` [PATCH pynfs 15/17] nfs41 svr: " Weston Andros Adamson
2014-06-04 21:02 ` [PATCH pynfs 16/17] rpc: on socket error, close and mark pipe inactive Weston Andros Adamson
2014-06-04 21:02 ` [PATCH pynfs 17/17] nfs3clnt: reconnect when sending on inactive pipe Weston Andros Adamson
     [not found] ` <1401915726-29092-6-git-send-email-dros@primarydata.com>
2014-06-05  2:31   ` [PATCH pynfs 05/17] move .x files to subdir 'xdrdef' J. Bruce Fields
2014-06-05 12:51     ` Weston Andros Adamson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.