linux-nfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Chandler <admin@genome.arizona.edu>
To: "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: NFS hangs on one interface
Date: Tue, 29 Oct 2019 23:30:59 -0700	[thread overview]
Message-ID: <82ee292f-f126-9e9f-d023-deb72d1a3971@genome.arizona.edu> (raw)
In-Reply-To: <YTBPR01MB2845B12E9C59F837FE35F40DDD650@YTBPR01MB2845.CANPRD01.PROD.OUTLOOK.COM>

Does this tcpdump help at all?  I ran:
tcpdump -i eth2 -s 0 -w out.pcap host x.4 and x.2

Then I tried "mount -v x.2:/data /data" in another term, waited until after the timeout then ^C then killed the tcpdump


   1   0.000000 x.4 -> x.2 TCP 66 739 > nfs [ACK] Seq=1 Ack=1 Win=140 Len=0 TSval=2837501537 TSecr=1577364421
   2   0.000034 x.4 -> x.2 NFS 206 [TCP Previous segment not captured] V3 READ Call, FH:0x48bf584a Offset:0 Len:131072
   3   0.000127 x.2 -> x.4 TCP 60 nfs > 739 [RST] Seq=1 Win=0 Len=0
   4   0.000148 x.2 -> x.4 TCP 60 nfs > 739 [RST] Seq=1 Win=0 Len=0
   5   3.000003 x.4 -> x.2 TCP 74 [TCP Port numbers reused] 739 > nfs [SYN] Seq=0 Win=17920 Len=0 MSS=8960 SACK_PERM=1 TSval=2837504537 TSecr=0 WS=128
   6   3.000182 x.2 -> x.4 TCP 74 nfs > 739 [SYN, ACK] Seq=0 Ack=1 Win=17896 Len=0 MSS=8960 SACK_PERM=1 TSval=1578327421 TSecr=2837504537 WS=128
   7   3.000205 x.4 -> x.2 TCP 66 739 > nfs [ACK] Seq=1 Ack=1 Win=17920 Len=0 TSval=2837504537 TSecr=1578327421
   8   3.000228 x.4 -> x.2 NFS 206 V3 READ Call, FH:0x48bf584a Offset:0 Len:131072
   9   3.000261 x.2 -> x.4 TCP 66 nfs > 739 [ACK] Seq=1 Ack=141 Win=19072 Len=0 TSval=1578327421 TSecr=2837504537
  10   4.100351 x.4 -> x.2 NFS 194 V4 Call PUTROOTFH | GETATTR
  11   4.139630 x.2 -> x.4 TCP 66 nfs > 798 [ACK] Seq=4113 Ack=129 Win=157 Len=0 TSval=1578328561 TSecr=2837505637
  12  44.294611 x.2 -> x.4 TCP 66 netconfsoaphttp > nfs [ACK] Seq=1 Ack=1 Win=140 Len=0 TSval=1578368716 TSecr=2837365831
  13  44.294628 x.4 -> x.2 TCP 66 [TCP ACKed unseen segment] nfs > netconfsoaphttp [ACK] Seq=3969 Ack=2 Win=149 Len=0 TSval=2837545831 TSecr=1578188716
  14  44.294634 x.2 -> x.4 TCP 66 [TCP Previous segment not captured] netconfsoaphttp > nfs [FIN, ACK] Seq=2 Ack=1 Win=140 Len=0 TSval=1578368716 TSecr=2837365831
  15  44.294688 x.4 -> x.2 TCP 66 [TCP ACKed unseen segment] nfs > netconfsoaphttp [FIN, ACK] Seq=3969 Ack=3 Win=149 Len=0 TSval=2837545831 TSecr=1578368716
  16  44.294734 x.2 -> x.4 TCP 60 netconfsoaphttp > nfs [RST] Seq=3 Win=0 Len=0
  17  47.294699 x.2 -> x.4 TCP 74 [TCP Port numbers reused] netconfsoaphttp > nfs [SYN] Seq=0 Win=17920 Len=0 MSS=8960 SACK_PERM=1 TSval=1578371716 TSecr=0 WS=128
  18  47.294723 x.4 -> x.2 TCP 74 nfs > netconfsoaphttp [SYN, ACK] Seq=0 Ack=1 Win=17896 Len=0 MSS=8960 SACK_PERM=1 TSval=2837548831 TSecr=1578371716 WS=128
  19  47.294770 x.2 -> x.4 TCP 66 netconfsoaphttp > nfs [ACK] Seq=1 Ack=1 Win=17920 Len=0 TSval=1578371716 TSecr=2837548831
  20  47.294794 x.2 -> x.4 NFS 270 V4 Call READDIR FH:0xcb5e6e28
  21  47.294802 x.4 -> x.2 TCP 66 nfs > netconfsoaphttp [ACK] Seq=1 Ack=205 Win=19072 Len=0 TSval=2837548831 TSecr=1578371716
  22  47.294975 x.4 -> x.2 NFS 4034 V4 Reply (Call In 20) READDIR
  23  47.495918 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  24  47.897918 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  25  48.701955 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  26  50.309927 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  27  53.525953 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  28  59.957895 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  29  62.999962 x.4 -> x.2 TCP 66 [TCP Keep-Alive] 739 > nfs [ACK] Seq=140 Ack=1 Win=17920 Len=0 TSval=2837564537 TSecr=1578327421
  30  63.000082 x.2 -> x.4 TCP 66 [TCP Previous segment not captured] nfs > 739 [ACK] Seq=17897 Ack=141 Win=19072 Len=0 TSval=1578387421 TSecr=2837504537
  31  64.099972 x.4 -> x.2 TCP 66 798 > nfs [FIN, ACK] Seq=129 Ack=1 Win=140 Len=0 TSval=2837565637 TSecr=1578304804
  32  64.100147 x.2 -> x.4 NFS 326 V4 Reply (Call In 10) PUTROOTFH | GETATTR
  33  64.100174 x.4 -> x.2 TCP 54 798 > nfs [RST] Seq=130 Win=0 Len=0
  34  67.099990 x.4 -> x.2 TCP 74 [TCP Port numbers reused] 798 > nfs [SYN] Seq=0 Win=17920 Len=0 MSS=8960 SACK_PERM=1 TSval=2837568637 TSecr=0 WS=128
  35  67.100135 x.2 -> x.4 TCP 74 nfs > 798 [SYN, ACK] Seq=0 Ack=1 Win=17896 Len=0 MSS=8960 SACK_PERM=1 TSval=1578391521 TSecr=2837568637 WS=128
  36  67.100158 x.4 -> x.2 TCP 66 798 > nfs [ACK] Seq=1 Ack=1 Win=17920 Len=0 TSval=2837568637 TSecr=1578391521
  37  67.100181 x.4 -> x.2 NFS 238 V4 Call READDIR FH:0x0366982c
  38  67.100222 x.2 -> x.4 TCP 66 nfs > 798 [ACK] Seq=1 Ack=173 Win=19072 Len=0 TSval=1578391521 TSecr=2837568637
  39  67.100241 x.4 -> x.2 NFS 194 V4 Call PUTROOTFH | GETATTR
  40  67.100285 x.2 -> x.4 TCP 66 nfs > 798 [ACK] Seq=1 Ack=301 Win=20096 Len=0 TSval=1578391521 TSecr=2837568637
  41  67.100332 x.2 -> x.4 NFS 326 V4 Reply (Call In 39) PUTROOTFH | GETATTR
  42  67.100339 x.4 -> x.2 TCP 66 798 > nfs [ACK] Seq=301 Ack=261 Win=19072 Len=0 TSval=2837568637 TSecr=1578391521
  43  67.111864 x.4 -> x.2 NFS 198 V4 Call GETATTR FH:0x62d40c52
  44  67.111991 x.2 -> x.4 NFS 158 [TCP Previous segment not captured] V4 Reply (Call In 43) GETATTR
  45  67.112010 x.4 -> x.2 TCP 78 [TCP Dup ACK 43#1] 798 > nfs [ACK] Seq=433 Ack=261 Win=19072 Len=0 TSval=2837568649 TSecr=1578391521 SLE=4373 SRE=4465
  46  72.821950 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  47  98.549965 x.4 -> x.2 NFS 4034 [RPC duplicate of #22][TCP Retransmission] V4 Reply (Call In 20) READDIR
  48 107.294623 x.2 -> x.4 TCP 66 [TCP Keep-Alive] netconfsoaphttp > nfs [ACK] Seq=204 Ack=1 Win=17920 Len=0 TSval=1578431716 TSecr=2837548831
  49 107.294642 x.4 -> x.2 TCP 66 [TCP Keep-Alive ACK] nfs > netconfsoaphttp [ACK] Seq=3969 Ack=205 Win=19072 Len=0 TSval=2837608831 TSecr=1578371716
  50 122.999956 x.4 -> x.2 TCP 66 [TCP Keep-Alive] 739 > nfs [ACK] Seq=140 Ack=1 Win=17920 Len=0 TSval=2837624537 TSecr=1578327421
  51 123.000088 x.2 -> x.4 TCP 66 [TCP Keep-Alive ACK] nfs > 739 [ACK] Seq=17897 Ack=141 Win=19072 Len=0 TSval=1578447421 TSecr=2837504537
  52 123.354599 Solarfla_y -> Solarfla_z ARP 60 Who has x.4?  Tell x.2
  53 123.354613 Solarfla_z -> Solarfla_y ARP 42 x.4 is at 00:0f:53:z
  54 127.110922 x.4 -> x.2 TCP 78 798 > nfs [FIN, ACK] Seq=433 Ack=261 Win=19072 Len=0 TSval=2837628648 TSecr=1578391521 SLE=4373 SRE=4465
  55 127.111041 x.2 -> x.4 TCP 66 nfs > 798 [FIN, ACK] Seq=4465 Ack=434 Win=21120 Len=0 TSval=1578451532 TSecr=2837628648
  56 127.111068 x.4 -> x.2 TCP 54 798 > nfs [RST] Seq=434 Win=0 Len=0
  57 130.110999 x.4 -> x.2 TCP 74 [TCP Port numbers reused] 798 > nfs [SYN] Seq=0 Win=17920 Len=0 MSS=8960 SACK_PERM=1 TSval=2837631648 TSecr=0 WS=128
  58 130.111106 x.2 -> x.4 TCP 74 nfs > 798 [SYN, ACK] Seq=0 Ack=1 Win=17896 Len=0 MSS=8960 SACK_PERM=1 TSval=1578454532 TSecr=2837631648 WS=128
  59 130.111133 x.4 -> x.2 TCP 66 798 > nfs [ACK] Seq=1 Ack=1 Win=17920 Len=0 TSval=2837631648 TSecr=1578454532
  60 130.111146 x.4 -> x.2 NFS 238 V4 Call READDIR FH:0x0366982c
  61 130.111162 x.4 -> x.2 NFS 198 V4 Call GETATTR FH:0x62d40c52
  62 130.111198 x.2 -> x.4 TCP 66 nfs > 798 [ACK] Seq=1 Ack=173 Win=19072 Len=0 TSval=1578454532 TSecr=2837631648
  63 130.111208 x.2 -> x.4 TCP 66 nfs > 798 [ACK] Seq=1 Ack=305 Win=20096 Len=0 TSval=1578454532 TSecr=2837631648
  64 130.111290 x.2 -> x.4 NFS 158 V4 Reply (Call In 61) GETATTR
  65 130.111296 x.4 -> x.2 TCP 66 798 > nfs [ACK] Seq=305 Ack=93 Win=17920 Len=0 TSval=2837631648 TSecr=1578454532
  66 130.111367 x.4 -> x.2 NFS 202 V4 Call GETATTR FH:0x62d40c52
  67 130.111430 x.2 -> x.4 NFS 178 V4 Reply (Call In 66) GETATTR
  68 130.111463 x.4 -> x.2 NFS 198 V4 Call GETATTR FH:0x62d40c52
  69 130.111513 x.2 -> x.4 NFS 158 [TCP Previous segment not captured] V4 Reply (Call In 68) GETATTR
  70 130.111521 x.4 -> x.2 TCP 78 [TCP Dup ACK 68#1] 798 > nfs [ACK] Seq=573 Ack=205 Win=17920 Len=0 TSval=2837631648 TSecr=1578454532 SLE=4317 SRE=4409

  reply	other threads:[~2019-10-30  6:31 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-23  0:34 NFS hangs on one interface Chandler
2019-10-23 17:15 ` J. Bruce Fields
2019-10-24 23:40   ` Chandler
2019-10-25  0:37     ` Rick Macklem
2019-10-30  6:30       ` Chandler [this message]
2019-11-05  0:28         ` Chandler
2019-11-05 16:24           ` Olga Kornievskaia
     [not found]             ` <49472814-8dcd-4594-d48e-be4c1d9a8d8f@genome.arizona.edu>
2019-11-11 15:46               ` Olga Kornievskaia
2019-11-12 22:14                 ` Chandler
2019-11-20 18:43 ` Chandler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=82ee292f-f126-9e9f-d023-deb72d1a3971@genome.arizona.edu \
    --to=admin@genome.arizona.edu \
    --cc=linux-nfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).