From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758834AbcEFUU1 (ORCPT ); Fri, 6 May 2016 16:20:27 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:34111 "EHLO mail-wm0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758281AbcEFUU0 convert rfc822-to-8bit (ORCPT ); Fri, 6 May 2016 16:20:26 -0400 Subject: Re: [PATCH RFC 10/22] block, bfq: add full hierarchical scheduling and cgroups support Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Content-Type: text/plain; charset=windows-1252 From: Paolo Valente In-Reply-To: <571E7E63.6010103@linaro.org> Date: Fri, 6 May 2016 22:20:18 +0200 Cc: Jens Axboe , Fabio Checconi , Arianna Avanzini , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org Content-Transfer-Encoding: 8BIT Message-Id: <2D83ACA0-2513-4019-9A0D-3166FDDE7485@linaro.org> References: <1454364778-25179-1-git-send-email-paolo.valente@linaro.org> <1454364778-25179-11-git-send-email-paolo.valente@linaro.org> <20160211222824.GD3741@mtj.duckdns.org> <57174CA7.5000706@linaro.org> <20160422181321.GV7822@mtj.duckdns.org> <20160422184110.GX7822@mtj.duckdns.org> <20160422193221.GY7822@mtj.duckdns.org> <57F65679-CD8E-43BA-8C46-C165B4C20677@linaro.org> <20160425192436.GE7822@mtj.duckdns.org> <571E7E63.6010103@linaro.org> To: Tejun Heo X-Mailer: Apple Mail (2.1878.6) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Il giorno 25/apr/2016, alle ore 22:30, Paolo ha scritto: > Il 25/04/2016 21:24, Tejun Heo ha scritto: >> Hello, Paolo. >> > > Hi > >> On Sat, Apr 23, 2016 at 09:07:47AM +0200, Paolo Valente wrote: >>> There is certainly something I don’t know here, because I don’t >>> understand why there is also a workqueue containing root-group I/O >>> all the time, if the only process doing I/O belongs to a different >>> (sub)group. >> >> Hmmm... maybe metadata updates? >> > > That's what I thought in the first place. But one half or one third of > the IOs sounded too much for metadata (the percentage varies over time > during the test). And root-group IOs are apparently large. Here is an > excerpt from the output of > > grep -B 1 insert_request trace > > kworker/u8:4-116 [002] d... 124.349971: 8,0 I W 3903488 + 1024 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.349978: 8,0 m N cfq409A / insert_request > -- > kworker/u8:4-116 [002] d... 124.350770: 8,0 I W 3904512 + 1200 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.350780: 8,0 m N cfq96A /seq_write insert_request > -- > kworker/u8:4-116 [002] d... 124.363911: 8,0 I W 3905712 + 1888 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.363916: 8,0 m N cfq409A / insert_request > -- > kworker/u8:4-116 [002] d... 124.364467: 8,0 I W 3907600 + 352 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.364474: 8,0 m N cfq96A /seq_write insert_request > -- > kworker/u8:4-116 [002] d... 124.369435: 8,0 I W 3907952 + 1680 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.369439: 8,0 m N cfq96A /seq_write insert_request > -- > kworker/u8:4-116 [002] d... 124.369441: 8,0 I W 3909632 + 560 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.369442: 8,0 m N cfq96A /seq_write insert_request > -- > kworker/u8:4-116 [002] d... 124.373299: 8,0 I W 3910192 + 1760 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.373301: 8,0 m N cfq409A / insert_request > -- > kworker/u8:4-116 [002] d... 124.373519: 8,0 I W 3911952 + 480 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.373522: 8,0 m N cfq96A /seq_write insert_request > -- > kworker/u8:4-116 [002] d... 124.381936: 8,0 I W 3912432 + 1728 [kworker/u8:4] > kworker/u8:4-116 [002] d... 124.381937: 8,0 m N cfq409A / insert_request > > >>> Anyway, if this is expected, then there is no reason to bother you >>> further on it. In contrast, the actual problem I see is the >>> following. If one third or half of the bios belong to a different >>> group than the writer that one wants to isolate, then, whatever >>> weight is assigned to the writer group, we will never be able to let >>> the writer get the desired share of the time (or of the bandwidth >>> with bfq and all quasi-sequential workloads). For instance, in the >>> scenario that you told me to try, the writer will never get 50% of >>> the time, with any scheduler. Am I missing something also on this? >> >> While a worker may jump across different cgroups, the IOs are still >> coming from somewhere and if the only IO generator on the machine is >> the test dd, the bios from that cgroup should dominate the IOs. I >> think it'd be helpful to investigate who's issuing the root cgroup >> IOs. >> > I can now confirm that, because of a little bug, a fraction ranging from one third to half of the writeback bios for the writer is wrongly associated with the root group. I'm sending a bugfix. I'm retesting BFQ after this blk fix. If I understand correctly, now you agree that BFQ is well suited for cgroups too, at least in principle. So I will apply all your suggestions and corrections, and submit a fresh patchset. Thanks, Paolo > Ok (if there is some quick way to get this information without > instrumenting the code, then any suggestion or pointer is welcome). > > Thanks, > Paolo > >> Thanks.