linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: <kim-btrfs@bluemoose.org.uk>
To: <linux-btrfs@vger.kernel.org>
Subject: BTRFS hangs - possibly NFS related?
Date: Tue, 1 Apr 2014 13:56:06 +0100	[thread overview]
Message-ID: <019301cf4da9$bf837930$3e8a6b90$@bluemoose.org.uk> (raw)

Apologies if this is known, but I've been lurking a while on the list and
not seen anything similar - and I'm running out of ideas on what to do next
to debug it.

Small HP microserver box, running Debian, EXT4 system disk plus 4 disk BTRFS
array shared over NFS (nfs-kernel-server) and SMB - the disks recently moved
from a different box where they've been running faultlessly for months,
although that didn't use NFS.

Under reasonable combined NFS and SMB load with only a couple of clients,
the shares lock up, load average on server and clients goes high and stays
high (10-12) and stays there.     Apparently not actually CPU and there's
little if any disk activity on the server.

Killing NFS and/or Samba sometimes helps, but it's always back when the load
comes back on. Chased round NFS and Samba options, then find that when the
clients hang it's unresponsive on the server directly to the disk.

Notice  a "btrfs-transacti" process hung in "d".    As are all the NFS
processes:

3779 ?        S<     0:00 [nfsd4]
3780 ?        S<     0:00 [nfsd4_callbacks]
3782 ?        D      0:27 [nfsd]
3783 ?        D      0:27 [nfsd]
3784 ?        D      0:28 [nfsd]
3785 ?        D      0:26 [nfsd]

"sync" instantly unsticks everything and it all works again for another
couple of minutes, when it locks up again, same symptoms.     Nothing
apparently written to kern.log or dmesg, which has been the frustration all
through - I don't know where to find the culprit!

As a band-aid I've put 
btrfs filesystem sync /mnt/btrfs

In the crontab once a minute which is actually working just fine  and has
been all morning - every 5 minutes was not enough.

Any recommendations on where I can look next, or any known holes I've fallen
in.?  Do I need to force NFS clients to sync in their mount options?


Background:
Kernel - 3.13-1-amd64 #1 SMP Debian 3.13.7-1 (2014-03-25)    AMD N54L with
10GB RAM.

##################################################
        Total devices 4 FS bytes used 848.88GiB
        devid    2 size 465.76GiB used 319.03GiB path /dev/sdc
        devid    4 size 465.76GiB used 319.00GiB path /dev/sda
        devid    5 size 455.76GiB used 309.03GiB path /dev/sdb2
        devid    6 size 931.51GiB used 785.00GiB path /dev/sdd

##################################################
Data, RAID1: total=864.00GiB, used=847.86GiB
System, RAID1: total=32.00MiB, used=128.00KiB
Metadata, RAID1: total=2.00GiB, used=1009.93MiB

A "scrub" passes without finding any errors.      

There are a couple of VM images with light traffic which do fragment a
little but I manually defrag those every day so often and I haven't had any
problems there - it certainly isn't thrashing.



Cheers
Kim


             reply	other threads:[~2014-04-01 12:56 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-04-01 12:56 kim-btrfs [this message]
2014-04-02  6:58 ` BTRFS hangs - possibly NFS related? Duncan
2014-05-25 11:42   ` kim-btrfs
2014-05-25 12:36     ` Chris Samuel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='019301cf4da9$bf837930$3e8a6b90$@bluemoose.org.uk' \
    --to=kim-btrfs@bluemoose.org.uk \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).