[Lustre-devel] Lustre RPC visualization

[Michael Kluge <Michael.Kluge@tu-dresden.de>]

> 00000100:00100000:0:1197446488.301126:0:28197:0:(service.c:721:ptlrpc_server_handle_request()) Handled RPC pname:cluuid+ref:pi
> d:xid:nid:opc ll_ost_26:lustre-MDT0000-mdtlov_UUID+5:27309:266:12345-0 at lo:400
> >>
> so..
> ll_ost_26 - thread name, which handle request,
> lustre-MDT0000-mdtlov_UUID - client which send request.
> +5 - number references to that request (unneed for you).
> 27309 - pid which handle request (or which send on client)
> 266 - xid - internal request id.
> 12345-0 - sender NID
> @lo - LND used to deliver request = @lo -> loopback, tcp -> tcp, @o2ib,  - IB hardware, @ptl - cray portals.
> 400 - command, you can look to enums "*_cmd_t " to decode that command identifier (in that example - 400 OBD_PING)

OK, thanks. I'll need to think about this as it might make sense to have one line per thread and not per UUID? Or is there already a 1:1 mapping?

> as other suggestions fix scaling, at you example only few clients has 100 requests in flight, but legend on bottom show up to 510.

Yes, did that while you were writing the mail ;) Definitely looks much better.



Michael

> 
> 
> On Jun 23, 2010, at 15:50, Michael Kluge wrote:
> 
>> Hi Alexey,
>> 
>> thanks  for the note. How do I do this if I only have a debug log? I see a field "cluuid+ref" in those lines of the log file that we are interested in. cluuid is probably "client uuid". This UUID is definitely associated to the node, that also owns the NID that appears in the same log line? What does "+ref" stands for?
>> 
>> 
>> Michael
>> 
>> Am 23.06.2010 um 12:29 schrieb Alexey Lyashkov:
>> 
>>> I think better to use client UUID instead of NID as client identification. Because in your's case - you can't separate info from two clients which run on same node.
>>> 
>>> 
>>> On Jun 22, 2010, at 19:12, Michael Kluge wrote:
>>> 
>>>> The remaining problems of the counter calculations have been fixed. There is a screenshot attached showing some values. The code is in a gforge server that we operate here in Dresden (gforge.zih.tu-dresden.de). The converter runs on Linux and on MAC OS X and you need Vampir to take a look at the OTF trace file.
>>>> 
>>>> Eric, Galen, for the moment I think I am done with the stuff I promised you at LUG this year? Are there any more ideas?
>>>> 
>>>> 
>>>> Michael
>>>> 
>>>> 
>>>> 
>>>> <lustre3.png>
>>>> 
>>>> 
>>>> 
>>>> 
>>>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>>> 
>>> 
>>> 
>>> --------------------------------------
>>> Alexey Lyashkov
>>> alexey.lyashkov at clusterstor.com
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> -- 
>> 
>> Michael Kluge, M.Sc.
>> 
>> Technische Universit?t Dresden
>> Center for Information Services and
>> High Performance Computing (ZIH)
>> D-01062 Dresden
>> Germany
>> 
>> Contact:
>> Willersbau, Room WIL A 208
>> Phone:  (+49) 351 463-34217
>> Fax:    (+49) 351 463-37773
>> e-mail: michael.kluge at tu-dresden.de
>> WWW:    http://www.tu-dresden.de/zih
>> 
> 


-- 

Michael Kluge, M.Sc.

Technische Universit?t Dresden
Center for Information Services and
High Performance Computing (ZIH)
D-01062 Dresden
Germany

Contact:
Willersbau, Room WIL A 208
Phone:  (+49) 351 463-34217
Fax:    (+49) 351 463-37773
e-mail: michael.kluge at tu-dresden.de
WWW:    http://www.tu-dresden.de/zih

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20100623/c0c14636/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: client_rpcs_in_flight.jpg
Type: image/jpg
Size: 326747 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20100623/c0c14636/attachment.jpg>