From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 910F2C25B08 for ; Wed, 17 Aug 2022 14:53:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237432AbiHQOxr (ORCPT ); Wed, 17 Aug 2022 10:53:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236390AbiHQOxq (ORCPT ); Wed, 17 Aug 2022 10:53:46 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA82F54675 for ; Wed, 17 Aug 2022 07:53:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660748025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SqR01pq9tq6qm7V2NPRinZ2qHQbpsqAKxq332tGf5Q4=; b=VtSIalHXdp4Eawl/QEcvZXID90iUywlXRWuk9WKAYqus+zs7k6qM6kiBPvEmLbN7dPZGz8 Dvk08GupVx1DcEVHpYv0Nq7lq7TyQl1AOIZlv65lXJLZXaSYJLDJFQhXoFKHT8DcRol9i9 0TEOnajLRSd/wrxWnnab50sA9lQqCfQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-588-DvkFF6R-PC-vl_GTPJvKdA-1; Wed, 17 Aug 2022 10:53:41 -0400 X-MC-Unique: DvkFF6R-PC-vl_GTPJvKdA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2A316185A7B2; Wed, 17 Aug 2022 14:53:41 +0000 (UTC) Received: from T590 (ovpn-8-20.pek2.redhat.com [10.72.8.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3C9D9C15BB3; Wed, 17 Aug 2022 14:53:34 +0000 (UTC) Date: Wed, 17 Aug 2022 22:53:29 +0800 From: Ming Lei To: Chris Murphy Cc: Nikolay Borisov , Jens Axboe , Jan Kara , Paolo Valente , Btrfs BTRFS , Linux-RAID , linux-block , linux-kernel , Josef Bacik Subject: Re: stalling IO regression since linux 5.12, through 5.18 Message-ID: References: <61e5ccda-a527-4fea-9850-91095ffa91c4@www.fastmail.com> <4995baed-c561-421d-ba3e-3a75d6a738a3@www.fastmail.com> <2b8a38fa-f15f-45e8-8caa-61c5f8cd52de@www.fastmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Wed, Aug 17, 2022 at 10:34:38AM -0400, Chris Murphy wrote: > > > On Wed, Aug 17, 2022, at 8:06 AM, Ming Lei wrote: > > > blk-mq debugfs log is usually helpful for io stall issue, care to post > > the blk-mq debugfs log: > > > > (cd /sys/kernel/debug/block/$disk && find . -type f -exec grep -aH . {} \;) > > This is only sda > https://drive.google.com/file/d/1aAld-kXb3RUiv_ShAvD_AGAFDRS03Lr0/view?usp=sharing >From the log, there isn't any in-flight IO request. So please confirm that it is collected after the IO stall is triggered. If yes, the issue may not be related with BFQ, and should be related with blk-cgroup code. Thanks, Ming