linux-cifs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Martijn de Gouw <martijn.de.gouw@prodrive-technologies.com>
To: <linux-cifs@vger.kernel.org>
Subject: Many processes end up in uninterruptible sleep accessing cifs mounts
Date: Wed, 3 Jul 2019 20:12:01 +0200	[thread overview]
Message-ID: <684ed01c-cbca-2716-bc28-b0a59a0f8521@prodrive-technologies.com> (raw)

Hi,

On our production servers, we have a lot of issues with cifs mounts.
All mounts are mounted via the dfs shares on our domain controller.
We have mounts using sec=krb5, sec=ntlmssp and sec=krb5,multiuser

All mounts are vers=3.0.

One of the symptoms is that our monitoring system complains about not 
being able to stat() every now and then, the next scraping cycle, stat() 
works again. Even when the mounts are not accesses at all.

Also, lot of applications get stuck on either accessing data on the 
mounts, or performing stat() like operations on the mounts.

For us, the worst part is that applications end up in 'D'. The number of 
'D' processes pile up really quickly, blocking users from performing 
their work.

We are running Linux 4.20.17 SMP PREEMPT on all machines. We tried 
upgrading to > 5.x, but caused even more problems and kernel hangs.

I do not really have a clue where to start debugging. I enabled kernel 
debug options suggested on the wiki, but the amount of logging is 
immense now.

Can you provide any pointers where to look or start debugging?
Or any help on how to kill those D processes and get our Linux servers 
stable again?

Regards, Martijn de Gouw
-- 
Martijn de Gouw
Designer
Prodrive Technologies
Mobile: +31 63 17 76 161
Phone:  +31 40 26 76 200

             reply	other threads:[~2019-07-03 18:21 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-03 18:12 Martijn de Gouw [this message]
2019-07-03 19:17 ` Many processes end up in uninterruptible sleep accessing cifs mounts Aurélien Aptel
2019-07-03 19:24   ` Steve French
     [not found]   ` <1fc4f6d0-6cdc-69a5-4359-23484d6bdfc9@prodrive-technologies.com>
2019-07-04 11:22     ` Aurélien Aptel
2019-07-04 13:36       ` Martijn de Gouw
2019-07-04 21:34         ` Pavel Shilovsky
2019-07-06 11:11           ` Martijn de Gouw
2019-07-06 13:05             ` Steve French
2019-07-06 16:40               ` Pavel Shilovsky
2019-07-09  8:48                 ` Martijn de Gouw
2019-07-15 23:40                   ` Pavel Shilovsky
2019-08-07  7:34                     ` Martijn de Gouw
2019-07-08  9:04               ` Martijn de Gouw
2019-07-08 11:06                 ` Aurélien Aptel
2019-07-08 13:18                   ` Martijn de Gouw
2019-07-04 21:08       ` Pavel Shilovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=684ed01c-cbca-2716-bc28-b0a59a0f8521@prodrive-technologies.com \
    --to=martijn.de.gouw@prodrive-technologies.com \
    --cc=linux-cifs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).