From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp73.iad3a.emailsrvr.com ([173.203.187.73]:51038 "EHLO smtp73.iad3a.emailsrvr.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752848AbdHTMeR (ORCPT ); Sun, 20 Aug 2017 08:34:17 -0400 Subject: Re: slow btrfs with a single kworker process using 100% CPU To: Stefan Priebe - Profihost AG Cc: "Konstantin V. Gavrilenko" , Roman Mamedov , linux-btrfs@vger.kernel.org, Peter Grandi References: <4772c3f2-0074-d86f-24c4-02ff0730fce7@rqc.ru> <064eaaed-7748-7064-874e-19d270d0854e@profihost.ag> <4669553.344.1502874134710.JavaMail.gkos@dynomob> <18522132.418.1502884115575.JavaMail.gkos@dynomob> <20170816170003.3f47321d@natsu> <31057849.442.1502886546489.JavaMail.gkos@dynomob> <27208cbf-0715-970c-5804-fdc7d65193ef@profihost.ag> From: Marat Khalili Message-ID: Date: Sun, 20 Aug 2017 15:34:13 +0300 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Sender: linux-btrfs-owner@vger.kernel.org List-ID: I'm a btrfs user, not a developer; developers can probably provide more detailed explanation by looking at stack traces in dmesg etc., but it's possible that there's just no quick fix (yet). I presume these are 1413 _full-volume_ snapshots. Then some operations have to process 43.65TiB*1413=62PiB of data -- well, metadata for that data, but it's still a lot as you may guess, especially if it's all heavily fragmented. You can either gradually reduce number of snapshots and wait (it may drag for weeks and months), or copy everything to a different device and reformat this one, then don't create that many snapshots again. As for "blocked for more than 120 seconds" messages in dmesg, I see them every night after I delete about a dozen of snapshots ~10TiB in _total_ volume, albeit with qgroups. These messages usually subside after about couple of hours. They only tell you what you already know: some btrfs operations are painfully slow. -- With Best Regards, Marat Khalili