From: Chris Murphy <lists@colorremedies.com>
To: Vojtech Myslivec <vojtech@xmyslivec.cz>
Cc: Chris Murphy <lists@colorremedies.com>,
Song Liu <songliubraving@fb.com>,
Michal Moravec <michal.moravec@logicworks.cz>,
Btrfs BTRFS <linux-btrfs@vger.kernel.org>,
Linux-RAID <linux-raid@vger.kernel.org>
Subject: Re: Linux RAID with btrfs stuck and consume 100 % CPU
Date: Thu, 17 Sep 2020 11:43:20 -0600 [thread overview]
Message-ID: <CAJCQCtTR1JZTLr+xTEZxrwh8xSfb+zpKjc+_S2tJGFofVMUdSQ@mail.gmail.com> (raw)
In-Reply-To: <5e79d1f8-7632-48ef-de56-9e79cba87434@xmyslivec.cz>
[Mon Aug 31 15:31:55 2020] sysrq: Show Blocked State
[Mon Aug 31 15:31:55 2020] task PC stack pid father
[Mon Aug 31 15:31:55 2020] md1_reclaim D 0 806 2 0x80004000
[Mon Aug 31 15:31:55 2020] Call Trace:
[Mon Aug 31 15:31:55 2020] __schedule+0x2dd/0x710
[Mon Aug 31 15:31:55 2020] ? finish_wait+0x80/0x80
[Mon Aug 31 15:31:55 2020] ? wbt_exit+0x30/0x30
[Mon Aug 31 15:31:55 2020] ? __wbt_done+0x30/0x30
[Mon Aug 31 15:31:55 2020] schedule+0x40/0xb0
[Mon Aug 31 15:31:55 2020] io_schedule+0x12/0x40
[Mon Aug 31 15:31:55 2020] rq_qos_wait+0xfa/0x170
[Mon Aug 31 15:31:55 2020] ? karma_partition+0x1e0/0x1e0
[Mon Aug 31 15:31:55 2020] ? wbt_exit+0x30/0x30
[Mon Aug 31 15:31:55 2020] wbt_wait+0x98/0xe0
[Mon Aug 31 15:31:55 2020] __rq_qos_throttle+0x23/0x30
[Mon Aug 31 15:31:55 2020] blk_mq_make_request+0x12a/0x5d0
[Mon Aug 31 15:31:55 2020] generic_make_request+0xcf/0x310
[Mon Aug 31 15:31:55 2020] submit_bio+0x42/0x1c0
[Mon Aug 31 15:31:55 2020] ? md_super_write.part.70+0x98/0x120 [md_mod]
[Mon Aug 31 15:31:55 2020] md_update_sb.part.71+0x3c0/0x8f0 [md_mod]
[Mon Aug 31 15:31:55 2020] r5l_do_reclaim+0x32a/0x3b0 [raid456]
[Mon Aug 31 15:31:55 2020] ? schedule_timeout+0x162/0x340
[Mon Aug 31 15:31:55 2020] ? prepare_to_wait_event+0xb6/0x140
[Mon Aug 31 15:31:55 2020] ? md_register_thread+0xd0/0xd0 [md_mod]
[Mon Aug 31 15:31:55 2020] md_thread+0x94/0x150 [md_mod]
[Mon Aug 31 15:31:55 2020] ? finish_wait+0x80/0x80
[Mon Aug 31 15:31:55 2020] kthread+0x112/0x130
[Mon Aug 31 15:31:55 2020] ? kthread_park+0x80/0x80
[Mon Aug 31 15:31:55 2020] ret_from_fork+0x22/0x40
*shrug*
These SSDs should be able to handle > 500MB/s. And > 130K IOPS. Swap
would have to be pretty heavy to slow down journal writes.
I'm not sure I have any good advise. My remaining ideas involve
changing configuration just to see if the problem goes away, rather
than actually understanding the cause of the problem.
---
Chris Murphy
next prev parent reply other threads:[~2020-09-17 17:44 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-22 20:47 Linux RAID with btrfs stuck and consume 100 % CPU Vojtech Myslivec
2020-07-22 22:00 ` antlists
2020-07-23 2:08 ` Chris Murphy
[not found] ` <29509e08-e373-b352-d696-fcb9f507a545@xmyslivec.cz>
2020-07-28 20:23 ` Chris Murphy
[not found] ` <695936b4-67a2-c862-9cb6-5545b4ab3c42@xmyslivec.cz>
2020-08-14 20:04 ` Chris Murphy
[not found] ` <2f2f1c21-c81b-55aa-6f77-e2d3f32d32cb@xmyslivec.cz>
2020-08-19 22:58 ` Chris Murphy
2020-08-19 23:11 ` Peter Grandi
2020-08-26 15:35 ` Vojtech Myslivec
2020-08-26 18:07 ` Chris Murphy
2020-09-16 9:42 ` Vojtech Myslivec
2020-09-17 17:08 ` Chris Murphy
2020-09-17 17:20 ` Chris Murphy
2020-09-17 17:43 ` Chris Murphy [this message]
2020-09-23 18:14 ` Vojtech Myslivec
[not found] ` <DBB07C8C-0D83-47DC-9B91-78AD385775E3@snapdragon.cc>
[not found] ` <D3026A55-A7F2-4432-87A8-3E9B2CACE4C2@snapdragon.cc>
[not found] ` <56AD80D0-6853-4E3A-A94C-AD1477D3FDA4@snapdragon.cc>
2021-03-17 15:55 ` Vojtech Myslivec
2020-07-29 21:06 ` Guoqing Jiang
2020-07-29 21:48 ` Chris Murphy
2020-08-12 14:19 ` Vojtech Myslivec
2020-07-30 6:45 ` Song Liu
2020-08-12 13:58 ` Vojtech Myslivec
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJCQCtTR1JZTLr+xTEZxrwh8xSfb+zpKjc+_S2tJGFofVMUdSQ@mail.gmail.com \
--to=lists@colorremedies.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=michal.moravec@logicworks.cz \
--cc=songliubraving@fb.com \
--cc=vojtech@xmyslivec.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).