nfsstat --sleep=#

nfsstat --sleep=#

[Kevin Constantine]

I'd really like to have a way to output the nfsstat counters at regular 
intervals (every 3 seconds) where the output is the difference between 3 
seconds ago and now.  Frequently I'll run a test and want to watch the 
nfs call profile throughout the course of the test.

Does something like this already exist?
Are there objections to seeing a feature like this?

I'm thinking something like:
nfsstat --sleep=1

  nfs v3 call:   Server   Client
        total:        0     3476
         null:        0        0
      getattr:        0     1679
      setattr:        0        0
       lookup:        0      839
       access:        0      839
     readlink:        0        0
         read:        0        0
        write:        0        0
       create:        0        0
        mkdir:        0        0
      symlink:        0        0
        mknod:        0        0
       remove:        0        0
        rmdir:        0        0
       rename:        0        0
         link:        0        0
      readdir:        0        0
  readdirplus:        0        0
       fsstat:        0      119
       fsinfo:        0        0
     pathconf:        0        0
       commit:        0        0

Re: nfsstat --sleep=#

[Greg Banks]

Kevin Constantine wrote:
> I'd really like to have a way to output the nfsstat counters at
> regular intervals (every 3 seconds) where the output is the difference
> between 3 seconds ago and now.  Frequently I'll run a test and want to
> watch the nfs call profile throughout the course of the test.
>
> Does something like this already exist?
> Are there objections to seeing a feature like this?

I wrote some patches a couple of years back to do that.  I'll see if I
can dig 'em up.

-- 
Greg Banks, P.Engineer, SGI Australian Software Group.
the brightly coloured sporks of revolution.
I don't speak for SGI.

Re: nfsstat --sleep=#

[Kevin Constantine]

It seems like everyone has a patch or a script that does this sort of 
thing.  It'd be great if it was just a part of nfsstat the way that 
vmstat, and iostat print differences on a regular basis.

Greg Banks wrote:
> Kevin Constantine wrote:
>> I'd really like to have a way to output the nfsstat counters at
>> regular intervals (every 3 seconds) where the output is the difference
>> between 3 seconds ago and now.  Frequently I'll run a test and want to
>> watch the nfs call profile throughout the course of the test.
>>
>> Does something like this already exist?
>> Are there objections to seeing a feature like this?
> 
> I wrote some patches a couple of years back to do that.  I'll see if I
> can dig 'em up.
> 

-- 
------------------------------------------------------------
Kevin Constantine

Systems Engineer		t: 818.460.8221
Walt Disney Animation Studios	e: kevin.constantine-P5ys19MLBK/QT0dZR+AlfA@public.gmane.org

Any sufficiently advanced technology is indistinguishable from magic.
     - Arthur C. Clarke

Re: nfsstat --sleep=#

[Chuck Lever]

Hi Kevin-

man watch(1)

On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:

> I'd really like to have a way to output the nfsstat counters at  
> regular intervals (every 3 seconds) where the output is the  
> difference between 3 seconds ago and now.  Frequently I'll run a  
> test and want to watch the nfs call profile throughout the course of  
> the test.
>
> Does something like this already exist?
> Are there objections to seeing a feature like this?
>
> I'm thinking something like:
> nfsstat --sleep=1
>
> nfs v3 call:   Server   Client
>       total:        0     3476
>        null:        0        0
>     getattr:        0     1679
>     setattr:        0        0
>      lookup:        0      839
>      access:        0      839
>    readlink:        0        0
>        read:        0        0
>       write:        0        0
>      create:        0        0
>       mkdir:        0        0
>     symlink:        0        0
>       mknod:        0        0
>      remove:        0        0
>       rmdir:        0        0
>      rename:        0        0
>        link:        0        0
>     readdir:        0        0
> readdirplus:        0        0
>      fsstat:        0      119
>      fsinfo:        0        0
>    pathconf:        0        0
>      commit:        0        0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: nfsstat --sleep=#

[J. Bruce Fields]

On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
> Hi Kevin-
>
> man watch(1)

What would you watch?

--b.

>
> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>
>> I'd really like to have a way to output the nfsstat counters at  
>> regular intervals (every 3 seconds) where the output is the difference 
>> between 3 seconds ago and now.  Frequently I'll run a test and want to 
>> watch the nfs call profile throughout the course of the test.
>>
>> Does something like this already exist?
>> Are there objections to seeing a feature like this?
>>
>> I'm thinking something like:
>> nfsstat --sleep=1
>>
>> nfs v3 call:   Server   Client
>>       total:        0     3476
>>        null:        0        0
>>     getattr:        0     1679
>>     setattr:        0        0
>>      lookup:        0      839
>>      access:        0      839
>>    readlink:        0        0
>>        read:        0        0
>>       write:        0        0
>>      create:        0        0
>>       mkdir:        0        0
>>     symlink:        0        0
>>       mknod:        0        0
>>      remove:        0        0
>>       rmdir:        0        0
>>      rename:        0        0
>>        link:        0        0
>>     readdir:        0        0
>> readdirplus:        0        0
>>      fsstat:        0      119
>>      fsinfo:        0        0
>>    pathconf:        0        0
>>      commit:        0        0
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"  
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> -- 
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfsstat --sleep=#

[Chuck Lever]

On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>> Hi Kevin-
>>
>> man watch(1)
>
> What would you watch?

For example:

   watch -n3 nfsstat -c

You can also use the "-d" option to highlight the differences between  
the current sample and the previous sample.

> --b.
>
>>
>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>
>>> I'd really like to have a way to output the nfsstat counters at
>>> regular intervals (every 3 seconds) where the output is the  
>>> difference
>>> between 3 seconds ago and now.  Frequently I'll run a test and  
>>> want to
>>> watch the nfs call profile throughout the course of the test.
>>>
>>> Does something like this already exist?
>>> Are there objections to seeing a feature like this?
>>>
>>> I'm thinking something like:
>>> nfsstat --sleep=1
>>>
>>> nfs v3 call:   Server   Client
>>>      total:        0     3476
>>>       null:        0        0
>>>    getattr:        0     1679
>>>    setattr:        0        0
>>>     lookup:        0      839
>>>     access:        0      839
>>>   readlink:        0        0
>>>       read:        0        0
>>>      write:        0        0
>>>     create:        0        0
>>>      mkdir:        0        0
>>>    symlink:        0        0
>>>      mknod:        0        0
>>>     remove:        0        0
>>>      rmdir:        0        0
>>>     rename:        0        0
>>>       link:        0        0
>>>    readdir:        0        0
>>> readdirplus:        0        0
>>>     fsstat:        0      119
>>>     fsinfo:        0        0
>>>   pathconf:        0        0
>>>     commit:        0        0
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>> in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> -- 
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux- 
>> nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: nfsstat --sleep=#

[Greg Banks]

Chuck Lever wrote:
> Hi Kevin-
>
> man watch(1)
Watch + today's nfsstat isn't smart enough to tell you deltas, or do
rate conversion.  However...

http://oss.sgi.com/projects/pcp/
 
PCP includes a gui tool which gives you rate conversion and graphical
presentation with history and automatic scaling of any metrics.  Plus
you can do archive recording and playback, and remote access. Supported
metrics include the existing server stats (but not the client-side
per-server stats).

In the absence of a useful tool like PCP, some time ago I did write a
patch add some of those features (rate conversion in particular) to
nfsstat.  The patch gives nfsstat a mode where it behaves like "vmstat
1" or "iostat 1", i.e. textually reports deltas or rates continuously in
a loop with a controllable interval.  I dug it up earlier today but
unfortunately it won't apply anymore (due to changes for the
implementation of the --sleep mode).  I won't have time to work on it
just now, but I can post it if anyone cares to mine it for ideas.

-- 
Greg Banks, P.Engineer, SGI Australian Software Group.
the brightly coloured sporks of revolution.
I don't speak for SGI.

Re: nfsstat --sleep=#

[J. Bruce Fields]

On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>> Hi Kevin-
>>>
>>> man watch(1)
>>
>> What would you watch?
>
> For example:
>
>   watch -n3 nfsstat -c
>
> You can also use the "-d" option to highlight the differences between  
> the current sample and the previous sample.

He was asking for deltas; the above only gives cumulative totals.

There's no accurate one-line solution using the existing nfsstat
commandline, but it should be easy to add.

--b.

>
>> --b.
>>
>>>
>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>
>>>> I'd really like to have a way to output the nfsstat counters at
>>>> regular intervals (every 3 seconds) where the output is the  
>>>> difference
>>>> between 3 seconds ago and now.  Frequently I'll run a test and  
>>>> want to
>>>> watch the nfs call profile throughout the course of the test.
>>>>
>>>> Does something like this already exist?
>>>> Are there objections to seeing a feature like this?
>>>>
>>>> I'm thinking something like:
>>>> nfsstat --sleep=1
>>>>
>>>> nfs v3 call:   Server   Client
>>>>      total:        0     3476
>>>>       null:        0        0
>>>>    getattr:        0     1679
>>>>    setattr:        0        0
>>>>     lookup:        0      839
>>>>     access:        0      839
>>>>   readlink:        0        0
>>>>       read:        0        0
>>>>      write:        0        0
>>>>     create:        0        0
>>>>      mkdir:        0        0
>>>>    symlink:        0        0
>>>>      mknod:        0        0
>>>>     remove:        0        0
>>>>      rmdir:        0        0
>>>>     rename:        0        0
>>>>       link:        0        0
>>>>    readdir:        0        0
>>>> readdirplus:        0        0
>>>>     fsstat:        0      119
>>>>     fsinfo:        0        0
>>>>   pathconf:        0        0
>>>>     commit:        0        0
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>> in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> -- 
>>> Chuck Lever
>>> chuck[dot]lever[at]oracle[dot]com
>>>
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" 
>>> in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> -- 
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfsstat --sleep=#

[Chuck Lever]

On Mar 12, 2009, at Mar 12, 2009, 12:24 PM, J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>> Hi Kevin-
>>>>
>>>> man watch(1)
>>>
>>> What would you watch?
>>
>> For example:
>>
>>  watch -n3 nfsstat -c
>>
>> You can also use the "-d" option to highlight the differences between
>> the current sample and the previous sample.
>
> He was asking for deltas; the above only gives cumulative totals.

In that case, he might try one of the Python scripts I wrote that  
report NFS and RPC metrics.  mountstats and nfs-iostat are included in  
the latest nfs-utils tarball.  If they don't do exactly what is  
needed, it should be easy to hack them.

> There's no accurate one-line solution using the existing nfsstat
> commandline, but it should be easy to add.
>
> --b.
>
>>
>>> --b.
>>>
>>>>
>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>>
>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>> regular intervals (every 3 seconds) where the output is the
>>>>> difference
>>>>> between 3 seconds ago and now.  Frequently I'll run a test and
>>>>> want to
>>>>> watch the nfs call profile throughout the course of the test.
>>>>>
>>>>> Does something like this already exist?
>>>>> Are there objections to seeing a feature like this?
>>>>>
>>>>> I'm thinking something like:
>>>>> nfsstat --sleep=1
>>>>>
>>>>> nfs v3 call:   Server   Client
>>>>>     total:        0     3476
>>>>>      null:        0        0
>>>>>   getattr:        0     1679
>>>>>   setattr:        0        0
>>>>>    lookup:        0      839
>>>>>    access:        0      839
>>>>>  readlink:        0        0
>>>>>      read:        0        0
>>>>>     write:        0        0
>>>>>    create:        0        0
>>>>>     mkdir:        0        0
>>>>>   symlink:        0        0
>>>>>     mknod:        0        0
>>>>>    remove:        0        0
>>>>>     rmdir:        0        0
>>>>>    rename:        0        0
>>>>>      link:        0        0
>>>>>   readdir:        0        0
>>>>> readdirplus:        0        0
>>>>>    fsstat:        0      119
>>>>>    fsinfo:        0        0
>>>>>  pathconf:        0        0
>>>>>    commit:        0        0
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux- 
>>>>> nfs"
>>>>> in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>> -- 
>>>> Chuck Lever
>>>> chuck[dot]lever[at]oracle[dot]com
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux- 
>>>> nfs"
>>>> in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> -- 
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux- 
>> nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: nfsstat --sleep=#

[Kevin Constantine]

J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>> Hi Kevin-
>>>>
>>>> man watch(1)
>>> What would you watch?
>> For example:
>>
>>   watch -n3 nfsstat -c
>>
>> You can also use the "-d" option to highlight the differences between  
>> the current sample and the previous sample.
> 
> He was asking for deltas; the above only gives cumulative totals.
> 
> There's no accurate one-line solution using the existing nfsstat
> commandline, but it should be easy to add.
> 

Something like this sort of works:
watch -n 1 'nfsstat --since /tmp/stats; nfsstat >/tmp/stats', but it 
feels more like a workaround than a feature.

Using watch doesn't allow you to see what happened in the past either.

Moving to a listed output format instead of the traditional nfsstat 
output (as seen below) makes it trivial with a simple grep to watch the 
stats that you really care about and ignore the rest.

-kevin


> --b.
> 
>>> --b.
>>>
>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>>
>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>> regular intervals (every 3 seconds) where the output is the  
>>>>> difference
>>>>> between 3 seconds ago and now.  Frequently I'll run a test and  
>>>>> want to
>>>>> watch the nfs call profile throughout the course of the test.
>>>>>
>>>>> Does something like this already exist?
>>>>> Are there objections to seeing a feature like this?
>>>>>
>>>>> I'm thinking something like:
>>>>> nfsstat --sleep=1
>>>>>
>>>>> nfs v3 call:   Server   Client
>>>>>      total:        0     3476
>>>>>       null:        0        0
>>>>>    getattr:        0     1679
>>>>>    setattr:        0        0
>>>>>     lookup:        0      839
>>>>>     access:        0      839
>>>>>   readlink:        0        0
>>>>>       read:        0        0
>>>>>      write:        0        0
>>>>>     create:        0        0
>>>>>      mkdir:        0        0
>>>>>    symlink:        0        0
>>>>>      mknod:        0        0
>>>>>     remove:        0        0
>>>>>      rmdir:        0        0
>>>>>     rename:        0        0
>>>>>       link:        0        0
>>>>>    readdir:        0        0
>>>>> readdirplus:        0        0
>>>>>     fsstat:        0      119
>>>>>     fsinfo:        0        0
>>>>>   pathconf:        0        0
>>>>>     commit:        0        0
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>>> in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> -- 
>>>> Chuck Lever
>>>> chuck[dot]lever[at]oracle[dot]com
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" 
>>>> in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> -- 
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfsstat --sleep=#

[J. Bruce Fields]

On Thu, Mar 12, 2009 at 09:45:04AM -0700, Kevin Constantine wrote:
> J. Bruce Fields wrote:
>> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>>> Hi Kevin-
>>>>>
>>>>> man watch(1)
>>>> What would you watch?
>>> For example:
>>>
>>>   watch -n3 nfsstat -c
>>>
>>> You can also use the "-d" option to highlight the differences between 
>>>  the current sample and the previous sample.
>>
>> He was asking for deltas; the above only gives cumulative totals.
>>
>> There's no accurate one-line solution using the existing nfsstat
>> commandline, but it should be easy to add.
>>
>
> Something like this sort of works:
> watch -n 1 'nfsstat --since /tmp/stats; nfsstat >/tmp/stats', but it  
> feels more like a workaround than a feature.

It's also slightly inaccurate, since it misses any activity that
happened in the brief time between the two invocations.

> Using watch doesn't allow you to see what happened in the past either.
>
> Moving to a listed output format instead of the traditional nfsstat  
> output (as seen below) makes it trivial with a simple grep to watch the  
> stats that you really care about and ignore the rest.

I think we'd all be happy to take patches.--b.


> -kevin
>
>
>> --b.
>>
>>>> --b.
>>>>
>>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>>>
>>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>>> regular intervals (every 3 seconds) where the output is the   
>>>>>> difference
>>>>>> between 3 seconds ago and now.  Frequently I'll run a test and  
>>>>>> want to
>>>>>> watch the nfs call profile throughout the course of the test.
>>>>>>
>>>>>> Does something like this already exist?
>>>>>> Are there objections to seeing a feature like this?
>>>>>>
>>>>>> I'm thinking something like:
>>>>>> nfsstat --sleep=1
>>>>>>
>>>>>> nfs v3 call:   Server   Client
>>>>>>      total:        0     3476
>>>>>>       null:        0        0
>>>>>>    getattr:        0     1679
>>>>>>    setattr:        0        0
>>>>>>     lookup:        0      839
>>>>>>     access:        0      839
>>>>>>   readlink:        0        0
>>>>>>       read:        0        0
>>>>>>      write:        0        0
>>>>>>     create:        0        0
>>>>>>      mkdir:        0        0
>>>>>>    symlink:        0        0
>>>>>>      mknod:        0        0
>>>>>>     remove:        0        0
>>>>>>      rmdir:        0        0
>>>>>>     rename:        0        0
>>>>>>       link:        0        0
>>>>>>    readdir:        0        0
>>>>>> readdirplus:        0        0
>>>>>>     fsstat:        0      119
>>>>>>     fsinfo:        0        0
>>>>>>   pathconf:        0        0
>>>>>>     commit:        0        0
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>>>> in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>> -- 
>>>>> Chuck Lever
>>>>> chuck[dot]lever[at]oracle[dot]com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>>> linux-nfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> -- 
>>> Chuck Lever
>>> chuck[dot]lever[at]oracle[dot]com
>>>
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfsstat --sleep=#

[Chuck Lever]

On Mar 12, 2009, at Mar 12, 2009, 12:45 PM, Kevin Constantine wrote:
> J. Bruce Fields wrote:
>> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>>> Hi Kevin-
>>>>>
>>>>> man watch(1)
>>>> What would you watch?
>>> For example:
>>>
>>>  watch -n3 nfsstat -c
>>>
>>> You can also use the "-d" option to highlight the differences  
>>> between  the current sample and the previous sample.
>> He was asking for deltas; the above only gives cumulative totals.
>> There's no accurate one-line solution using the existing nfsstat
>> commandline, but it should be easy to add.
>
> Something like this sort of works:
> watch -n 1 'nfsstat --since /tmp/stats; nfsstat >/tmp/stats', but it  
> feels more like a workaround than a feature.
>
> Using watch doesn't allow you to see what happened in the past either.
>
> Moving to a listed output format instead of the traditional nfsstat  
> output (as seen below) makes it trivial with a simple grep to watch  
> the stats that you really care about and ignore the rest.

It sounds like you might find the NFS/RPC metrics reported by  
mountstats and nfs-iostat are what you want.

Or, you can look at the raw data in /proc/self/mountstats.

> -kevin
>
>
>> --b.
>>>> --b.
>>>>
>>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine  
>>>>> wrote:
>>>>>
>>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>>> regular intervals (every 3 seconds) where the output is the   
>>>>>> difference
>>>>>> between 3 seconds ago and now.  Frequently I'll run a test and   
>>>>>> want to
>>>>>> watch the nfs call profile throughout the course of the test.
>>>>>>
>>>>>> Does something like this already exist?
>>>>>> Are there objections to seeing a feature like this?
>>>>>>
>>>>>> I'm thinking something like:
>>>>>> nfsstat --sleep=1
>>>>>>
>>>>>> nfs v3 call:   Server   Client
>>>>>>     total:        0     3476
>>>>>>      null:        0        0
>>>>>>   getattr:        0     1679
>>>>>>   setattr:        0        0
>>>>>>    lookup:        0      839
>>>>>>    access:        0      839
>>>>>>  readlink:        0        0
>>>>>>      read:        0        0
>>>>>>     write:        0        0
>>>>>>    create:        0        0
>>>>>>     mkdir:        0        0
>>>>>>   symlink:        0        0
>>>>>>     mknod:        0        0
>>>>>>    remove:        0        0
>>>>>>     rmdir:        0        0
>>>>>>    rename:        0        0
>>>>>>      link:        0        0
>>>>>>   readdir:        0        0
>>>>>> readdirplus:        0        0
>>>>>>    fsstat:        0      119
>>>>>>    fsinfo:        0        0
>>>>>>  pathconf:        0        0
>>>>>>    commit:        0        0
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux- 
>>>>>> nfs"
>>>>>> in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo- 
>>>>>> info.html
>>>>> -- 
>>>>> Chuck Lever
>>>>> chuck[dot]lever[at]oracle[dot]com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux- 
>>>>> nfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> -- 
>>> Chuck Lever
>>> chuck[dot]lever[at]oracle[dot]com
>>>
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux- 
>>> nfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: nfsstat --sleep=#

[Kevin Constantine]

J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 09:45:04AM -0700, Kevin Constantine wrote:
>> J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>>>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>>>> Hi Kevin-
>>>>>>
>>>>>> man watch(1)
>>>>> What would you watch?
>>>> For example:
>>>>
>>>>   watch -n3 nfsstat -c
>>>>
>>>> You can also use the "-d" option to highlight the differences between 
>>>>  the current sample and the previous sample.
>>> He was asking for deltas; the above only gives cumulative totals.
>>>
>>> There's no accurate one-line solution using the existing nfsstat
>>> commandline, but it should be easy to add.
>>>
>> Something like this sort of works:
>> watch -n 1 'nfsstat --since /tmp/stats; nfsstat >/tmp/stats', but it  
>> feels more like a workaround than a feature.
> 
> It's also slightly inaccurate, since it misses any activity that
> happened in the brief time between the two invocations.
> 
>> Using watch doesn't allow you to see what happened in the past either.
>>
>> Moving to a listed output format instead of the traditional nfsstat  
>> output (as seen below) makes it trivial with a simple grep to watch the  
>> stats that you really care about and ignore the rest.
> 
> I think we'd all be happy to take patches.--b.
> 
Cool.  I'll spend some time today then.


> 
>> -kevin
>>
>>
>>> --b.
>>>
>>>>> --b.
>>>>>
>>>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>>>>
>>>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>>>> regular intervals (every 3 seconds) where the output is the   
>>>>>>> difference
>>>>>>> between 3 seconds ago and now.  Frequently I'll run a test and  
>>>>>>> want to
>>>>>>> watch the nfs call profile throughout the course of the test.
>>>>>>>
>>>>>>> Does something like this already exist?
>>>>>>> Are there objections to seeing a feature like this?
>>>>>>>
>>>>>>> I'm thinking something like:
>>>>>>> nfsstat --sleep=1
>>>>>>>
>>>>>>> nfs v3 call:   Server   Client
>>>>>>>      total:        0     3476
>>>>>>>       null:        0        0
>>>>>>>    getattr:        0     1679
>>>>>>>    setattr:        0        0
>>>>>>>     lookup:        0      839
>>>>>>>     access:        0      839
>>>>>>>   readlink:        0        0
>>>>>>>       read:        0        0
>>>>>>>      write:        0        0
>>>>>>>     create:        0        0
>>>>>>>      mkdir:        0        0
>>>>>>>    symlink:        0        0
>>>>>>>      mknod:        0        0
>>>>>>>     remove:        0        0
>>>>>>>      rmdir:        0        0
>>>>>>>     rename:        0        0
>>>>>>>       link:        0        0
>>>>>>>    readdir:        0        0
>>>>>>> readdirplus:        0        0
>>>>>>>     fsstat:        0      119
>>>>>>>     fsinfo:        0        0
>>>>>>>   pathconf:        0        0
>>>>>>>     commit:        0        0
>>>>>>>
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>>>>> in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>> -- 
>>>>>> Chuck Lever
>>>>>> chuck[dot]lever[at]oracle[dot]com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>>>> linux-nfs" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> -- 
>>>> Chuck Lever
>>>> chuck[dot]lever[at]oracle[dot]com
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
------------------------------------------------------------
Kevin Constantine

Systems Engineer		t: 818.460.8221
Walt Disney Animation Studios	e: kevin.constantine-P5ys19MLBK/QT0dZR+AlfA@public.gmane.org

Any sufficiently advanced technology is indistinguishable from magic.
     - Arthur C. Clarke

Re: nfsstat --sleep=#

[Chuck Lever]

On Mar 12, 2009, at Mar 12, 2009, 12:48 PM, J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 09:45:04AM -0700, Kevin Constantine wrote:
>> J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>>>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>>>> Hi Kevin-
>>>>>>
>>>>>> man watch(1)
>>>>> What would you watch?
>>>> For example:
>>>>
>>>>  watch -n3 nfsstat -c
>>>>
>>>> You can also use the "-d" option to highlight the differences  
>>>> between
>>>> the current sample and the previous sample.
>>>
>>> He was asking for deltas; the above only gives cumulative totals.
>>>
>>> There's no accurate one-line solution using the existing nfsstat
>>> commandline, but it should be easy to add.
>>>
>>
>> Something like this sort of works:
>> watch -n 1 'nfsstat --since /tmp/stats; nfsstat >/tmp/stats', but it
>> feels more like a workaround than a feature.
>
> It's also slightly inaccurate, since it misses any activity that
> happened in the brief time between the two invocations.
>
>> Using watch doesn't allow you to see what happened in the past  
>> either.
>>
>> Moving to a listed output format instead of the traditional nfsstat
>> output (as seen below) makes it trivial with a simple grep to watch  
>> the
>> stats that you really care about and ignore the rest.
>
> I think we'd all be happy to take patches.--b.

Actually I would rather see the performance metrics scripts improved.   
These tools give a lot more information than nfsstat ever will be able  
to.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: nfsstat --sleep=#

[J. Bruce Fields]

On Thu, Mar 12, 2009 at 01:22:32PM -0400, Chuck Lever wrote:
> Actually I would rather see the performance metrics scripts improved.   
> These tools give a lot more information than nfsstat ever will be able  
> to.

Probably so, but those scripts are a bit hard to find, aren't they?

We should
	- get distributions to install them by default
	- write man pages?
	- add references to them where possible (from the nfsstat man
	  page, from howto's/faq's/?)

Until then, unfortunately, improvements to nfsstat are more useful,
since nfsstat is the thing people are more likely to run across.

(Which might be an argument just for adding their functionality into
nfsstat--that would have the one benefit that there'd be only one
command name users have to know about.)

--b.

Re: nfsstat --sleep=#

[Chuck Lever]

On Mar 12, 2009, at Mar 12, 2009, 3:32 PM, J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 01:22:32PM -0400, Chuck Lever wrote:
>> Actually I would rather see the performance metrics scripts improved.
>> These tools give a lot more information than nfsstat ever will be  
>> able
>> to.
>
> Probably so, but those scripts are a bit hard to find, aren't they?
>
> We should
> 	- get distributions to install them by default
> 	- write man pages?
> 	- add references to them where possible (from the nfsstat man
> 	  page, from howto's/faq's/?)

Steve promised me Red Hat would take care of this when these were  
added to nfs-utils last year.

> Until then, unfortunately, improvements to nfsstat are more useful,
> since nfsstat is the thing people are more likely to run across.

Again, I think improving nfsstat at this point (which is merely for  
compatibility with Solaris) would be wasted work, if we already have  
what is needed in another tool.  The Python tools are much more  
sophisticated, and it would be confusing to add their functionality to  
nfsstat (e.g. why can I zero the legacy stats with the -z option, but  
not the stats the come from /proc/self/mountstats?).

Let's spend the effort on the tools that give us deeper results.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: nfsstat --sleep=#

[Steve Dickson]

Chuck Lever wrote:
> On Mar 12, 2009, at Mar 12, 2009, 3:32 PM, J. Bruce Fields wrote:
>> On Thu, Mar 12, 2009 at 01:22:32PM -0400, Chuck Lever wrote:
>>> Actually I would rather see the performance metrics scripts improved.
>>> These tools give a lot more information than nfsstat ever will be able
>>> to.
>>
>> Probably so, but those scripts are a bit hard to find, aren't they?
>>
>> We should
>>     - get distributions to install them by default
>>     - write man pages?
>>     - add references to them where possible (from the nfsstat man
>>       page, from howto's/faq's/?)
> 
> Steve promised me Red Hat would take care of this when these were added
> to nfs-utils last year.
Not sure what I exactly promised (that's usually not my style 8-) )
but those scripts definitely fell off my radar...  

> 
>> Until then, unfortunately, improvements to nfsstat are more useful,
>> since nfsstat is the thing people are more likely to run across.
> 
> Again, I think improving nfsstat at this point (which is merely for
> compatibility with Solaris) would be wasted work, if we already have
> what is needed in another tool.  The Python tools are much more
> sophisticated, and it would be confusing to add their functionality to
> nfsstat (e.g. why can I zero the legacy stats with the -z option, but
> not the stats the come from /proc/self/mountstats?).
> 
> Let's spend the effort on the tools that give us deeper results.
I have to agree... I truly think there is a wealth of untapped information
in those mountstats... Just waiting for someone to dip them out...

steved.
 

Re: nfsstat --sleep=#

[Chuck Lever]

Hi Steve-

On Mar 16, 2009, at Mar 16, 2009, 12:18 PM, Steve Dickson wrote:
> Chuck Lever wrote:
>> On Mar 12, 2009, at Mar 12, 2009, 3:32 PM, J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 01:22:32PM -0400, Chuck Lever wrote:
>>>> Actually I would rather see the performance metrics scripts  
>>>> improved.
>>>> These tools give a lot more information than nfsstat ever will be  
>>>> able
>>>> to.
>>>
>>> Probably so, but those scripts are a bit hard to find, aren't they?
>>>
>>> We should
>>>    - get distributions to install them by default
>>>    - write man pages?
>>>    - add references to them where possible (from the nfsstat man
>>>      page, from howto's/faq's/?)
>>
>> Steve promised me Red Hat would take care of this when these were  
>> added
>> to nfs-utils last year.
> Not sure what I exactly promised (that's usually not my style 8-) )
> but those scripts definitely fell off my radar...

You promised to find an intern to compose man pages, and do the  
legwork needed to get these reviewed internally, and installed when  
nfs-utils is installed.  I think that last part requires constructing  
a suitable Makefile.am in the nfs-utils directory where these now  
live, and updating the parent Makefiles to invoke it.

There was some controversy over whether nfs-utils should have a  
dependency on having the Python interpreter installed.  Since Red Hat  
(for example) already uses Python for the system-config-foo utilities,  
this might not be too much of a problem.  In any event, if we decide  
that Python is not appropriate, it would be simple to convert the  
scripts to C, or even add this functionality to nfsstat (Bruce's  
suggestion).

>>> Until then, unfortunately, improvements to nfsstat are more useful,
>>> since nfsstat is the thing people are more likely to run across.
>>
>> Again, I think improving nfsstat at this point (which is merely for
>> compatibility with Solaris) would be wasted work, if we already have
>> what is needed in another tool.  The Python tools are much more
>> sophisticated, and it would be confusing to add their functionality  
>> to
>> nfsstat (e.g. why can I zero the legacy stats with the -z option, but
>> not the stats the come from /proc/self/mountstats?).
>>
>> Let's spend the effort on the tools that give us deeper results.
> I have to agree... I truly think there is a wealth of untapped  
> information
> in those mountstats... Just waiting for someone to dip them out...

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com