From: Timo Rothenpieler <timo@rothenpieler.org>
To: Olga Kornievskaia <aglo@umich.edu>
Cc: linux-rdma <linux-rdma@vger.kernel.org>,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: copy_file_range() infinitely hangs on NFSv4.2 over RDMA
Date: Thu, 18 Feb 2021 21:22:36 +0100 [thread overview]
Message-ID: <def12560-2481-b17d-5a42-7236edbd5392@rothenpieler.org> (raw)
In-Reply-To: <CAN-5tyGhyh0ZF77voaN4TNgMt+jSUG0PMp-KixfTvgUhDdhDUQ@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 1154 bytes --]
On 18.02.2021 19:30, Olga Kornievskaia wrote:
> Thank you for getting tracepoints from a busy server but can you get
> more? As suspected, the server is having issues sending the callback.
> I'm not sure why. Any chance to turn on the server's sunrpc
> tracespoints, probably both sunrpc and rdmas tracepoints, I wonder if
> we can any more info about why it's failing?
I isolated out two of the machines on that cluster now, one acting as
NFS server from an ext4 mount, the other is the same client as before.
That way I managed to capture a trace and ibdump of an entire cycle:
mount + successful copy + 5 minutes later a copy that got stuck
Next to no noise happened during those traces, you can find them attached.
Another observation made due to this: unmount and re-mounting the NFS
share also gets it back into working condition for a while, no reboot
necessary.
During this trace, I got "lucky", and after just 5 minutes of waiting,
it got stuck.
Before that, I had a run of mount + trying to copy every 5 minutes where
it ran for 45 minutes without getting stuck. At which point I decided to
remount once more.
[-- Attachment #1.2: sniffer.pcap.xz --]
[-- Type: application/octet-stream, Size: 11540 bytes --]
[-- Attachment #1.3: trace.dat.xz --]
[-- Type: application/octet-stream, Size: 566284 bytes --]
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4494 bytes --]
next prev parent reply other threads:[~2021-02-18 20:25 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-14 3:31 copy_file_range() infinitely hangs on NFSv4.2 over RDMA Timo Rothenpieler
2021-02-16 20:12 ` Olga Kornievskaia
2021-02-16 20:37 ` Timo Rothenpieler
2021-02-16 22:27 ` Timo Rothenpieler
2021-02-17 22:37 ` Olga Kornievskaia
2021-02-18 1:12 ` Timo Rothenpieler
2021-02-18 3:52 ` Olga Kornievskaia
2021-02-18 13:28 ` Timo Rothenpieler
2021-02-18 15:55 ` Timo Rothenpieler
2021-02-18 18:30 ` Olga Kornievskaia
2021-02-18 20:22 ` Timo Rothenpieler [this message]
2021-02-19 17:38 ` Olga Kornievskaia
2021-02-19 17:48 ` Chuck Lever
2021-02-19 18:01 ` Timo Rothenpieler
2021-02-19 18:48 ` Chuck Lever
2021-02-19 20:37 ` Timo Rothenpieler
2021-02-19 20:43 ` Olga Kornievskaia
2021-02-19 20:55 ` Chuck Lever
2021-02-20 21:03 ` Timo Rothenpieler
2021-02-21 17:45 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=def12560-2481-b17d-5a42-7236edbd5392@rothenpieler.org \
--to=timo@rothenpieler.org \
--cc=aglo@umich.edu \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.