From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F143AC31E49 for ; Thu, 13 Jun 2019 19:05:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D8EA020B7C for ; Thu, 13 Jun 2019 19:05:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727327AbfFMTFw (ORCPT ); Thu, 13 Jun 2019 15:05:52 -0400 Received: from outgoing-stata.csail.mit.edu ([128.30.2.210]:36525 "EHLO outgoing-stata.csail.mit.edu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727298AbfFMTFw (ORCPT ); Thu, 13 Jun 2019 15:05:52 -0400 Received: from [4.30.142.84] (helo=srivatsab-a01.vmware.com) by outgoing-stata.csail.mit.edu with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1hbV2i-0007Iv-Mi; Thu, 13 Jun 2019 15:05:48 -0400 Subject: Re: CFQ idling kills I/O performance on ext4 with blkio cgroup controller To: Jan Kara Cc: Paolo Valente , linux-fsdevel@vger.kernel.org, linux-block , linux-ext4@vger.kernel.org, cgroups@vger.kernel.org, kernel list , Jens Axboe , Jeff Moyer , Theodore Ts'o , amakhalov@vmware.com, anishs@vmware.com, srivatsab@vmware.com, Ulf Hansson , Linus Walleij , Greg Kroah-Hartman , Stable References: <5b71028c-72f0-73dd-0cd5-f28ff298a0a3@csail.mit.edu> <0d6e3c02-1952-2177-02d7-10ebeb133940@csail.mit.edu> <7B74A790-BD98-412B-ADAB-3B513FB1944E@linaro.org> <6a6f4aa4-fc95-f132-55b2-224ff52bd2d8@csail.mit.edu> <7c5e9d11-4a3d-7df4-c1e6-7c95919522ab@csail.mit.edu> <20190612130446.GD14578@quack2.suse.cz> <20190613082053.GD26505@quack2.suse.cz> From: "Srivatsa S. Bhat" Message-ID: Date: Thu, 13 Jun 2019 12:05:46 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: <20190613082053.GD26505@quack2.suse.cz> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 6/13/19 1:20 AM, Jan Kara wrote: > On Wed 12-06-19 12:36:53, Srivatsa S. Bhat wrote: >> >> [ Adding Greg to CC ] >> >> On 6/12/19 6:04 AM, Jan Kara wrote: >>> On Tue 11-06-19 15:34:48, Srivatsa S. Bhat wrote: >>>> On 6/2/19 12:04 AM, Srivatsa S. Bhat wrote: >>>>> On 5/30/19 3:45 AM, Paolo Valente wrote: >>>>>> >>>> [...] >>>>>> At any rate, since you pointed out that you are interested in >>>>>> out-of-the-box performance, let me complete the context: in case >>>>>> low_latency is left set, one gets, in return for this 12% loss, >>>>>> a) at least 1000% higher responsiveness, e.g., 1000% lower start-up >>>>>> times of applications under load [1]; >>>>>> b) 500-1000% higher throughput in multi-client server workloads, as I >>>>>> already pointed out [2]. >>>>>> >>>>> >>>>> I'm very happy that you could solve the problem without having to >>>>> compromise on any of the performance characteristics/features of BFQ! >>>>> >>>>> >>>>>> I'm going to prepare complete patches. In addition, if ok for you, >>>>>> I'll report these results on the bug you created. Then I guess we can >>>>>> close it. >>>>>> >>>>> >>>>> Sounds great! >>>>> >>>> >>>> Hi Paolo, >>>> >>>> Hope you are doing great! >>>> >>>> I was wondering if you got a chance to post these patches to LKML for >>>> review and inclusion... (No hurry, of course!) >>>> >>>> Also, since your fixes address the performance issues in BFQ, do you >>>> have any thoughts on whether they can be adapted to CFQ as well, to >>>> benefit the older stable kernels that still support CFQ? >>> >>> Since CFQ doesn't exist in current upstream kernel anymore, I seriously >>> doubt you'll be able to get any performance improvements for it in the >>> stable kernels... >>> >> >> I suspected as much, but that seems unfortunate though. The latest LTS >> kernel is based on 4.19, which still supports CFQ. It would have been >> great to have a process to address significant issues on older >> kernels too. > > Well, you could still tune the performance difference by changing > slice_idle and group_idle tunables for CFQ (in > /sys/block//queue/iosched/). Changing these to lower values will > reduce the throughput loss when switching between cgroups at the cost of > lower accuracy of enforcing configured IO proportions among cgroups. > Good point, and seems fair enough, thank you! Regards, Srivatsa VMware Photon OS