From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A3F0C43217 for ; Sat, 8 Jan 2022 04:54:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230128AbiAHEy0 (ORCPT ); Fri, 7 Jan 2022 23:54:26 -0500 Received: from mx.ewheeler.net ([173.205.220.69]:53842 "EHLO mx.ewheeler.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229527AbiAHEyY (ORCPT ); Fri, 7 Jan 2022 23:54:24 -0500 Received: from localhost (localhost [127.0.0.1]) by mx.ewheeler.net (Postfix) with ESMTP id 52E7581; Fri, 7 Jan 2022 20:54:24 -0800 (PST) X-Virus-Scanned: amavisd-new at ewheeler.net Received: from mx.ewheeler.net ([127.0.0.1]) by localhost (mx.ewheeler.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id wLW4KuNBqH4k; Fri, 7 Jan 2022 20:54:19 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx.ewheeler.net (Postfix) with ESMTPSA id 1A62840; Fri, 7 Jan 2022 20:54:18 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 mx.ewheeler.net 1A62840 Date: Fri, 7 Jan 2022 20:54:16 -0800 (PST) From: Eric Wheeler To: "Martin K. Petersen" cc: Coly Li , linux-block@vger.kernel.org, Jonathan Corbet , Kent Overstreet , "open list:DOCUMENTATION" , open list , "open list:BCACHE (BLOCK LAYER CACHE)" Subject: Re: [PATCH] bcache: make stripe_size configurable and persistent for hardware raid5/6 In-Reply-To: Message-ID: References: <1561245371-10235-1-git-send-email-bcache@lists.ewheeler.net> <200638b0-7cba-38b4-20c4-b325f3cfe862@suse.de> <8a9131dc-9bf7-a24a-f7b8-35e0c019e905@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 7 Jan 2022, Martin K. Petersen wrote: > Eric, > > > Even new new RAID controlers that _do_ provide `io_opt` still do _not_ > > indicate partial_stripes_expensive (which is an mdraid feature, but Martin > > please correct me if I'm wrong here). > > partial_stripes_expensive is a bcache thing, I am not sure why it needs > a separate flag. It is implied, although I guess one could argue that > RAID0 is a special case since partial writes are not as painful as with > parity RAID. I'm guessing bcache used did some optimization for queue->limits.raid_partial_stripes_expensive because md raid5 code sets this flag. At least when using Linux md as the RAID5 implementation it gets configured automatically: raid5.c: mddev->queue->limits.raid_partial_stripes_expensive = 1; https://elixir.bootlin.com/linux/latest/source/drivers/md/raid5.c#L7729 Interestingly only bcache uses it, but md does set it. > The SCSI spec states that submitting an I/O that is smaller than io_min > "may incur delays in processing the command". And similarly, submitting > a command larger than io_opt "may incur delays in processing the > command". > > IOW, the spec says "don't write less than an aligned multiple of the > stripe chunk size" and "don't write more than an aligned full > stripe". That leaves "aligned multiples of the stripe chunk size but > less than the full stripe width" unaccounted for. And I guess that's > what the bcache flag is trying to capture. Maybe any time io_opt is provided then partial_stripes_expensive should be flagged too and any code to the contrary should be removed? Question: Does anyone have a reason to keep partial_stripes_expensive in the kernel at all? > SCSI doesn't go into details about RAID levels and other implementation > details which is why the wording is deliberately vague. But obviously > the expectation is that partial stripe writes are slower than full. > > In my book any component in the stack that sees either io_min or io_opt > should try very hard to send I/Os that are aligned multiples of those > values. I am not opposed to letting users manually twiddle the > settings. But I do think that we should aim for the stack doing the > right thing when it sees io_opt reported on a device. Agreed, thanks for the feedback! -Eric > > -- > Martin K. Petersen Oracle Linux Engineering >