From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 007BCC43603 for ; Mon, 16 Dec 2019 09:28:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CA3AF20725 for ; Mon, 16 Dec 2019 09:28:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1576488503; bh=RW14kw0PJ2wrm6ulu6HtyD0f3CFUMop1Am8yugJWJLY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=b9xhueDbWi2lDrQVQGKlqgz6hnfOYg0x1P+XxTydQncJ+tLpUI6zPOK1szkZCHzeG aXHljoSrPGxLdXVfgx8MA2jQ4jKk1iLjLNF0IJngihyk69rjiLZUUeQzmAaxGRaxQv xmX3JjXFL51NOsPEHDFYMemw0Xpe/DAu4eG48Glc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727384AbfLPJ2W (ORCPT ); Mon, 16 Dec 2019 04:28:22 -0500 Received: from mail.kernel.org ([198.145.29.99]:39536 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727053AbfLPJ2W (ORCPT ); Mon, 16 Dec 2019 04:28:22 -0500 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BAF312075A; Mon, 16 Dec 2019 09:28:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1576488501; bh=RW14kw0PJ2wrm6ulu6HtyD0f3CFUMop1Am8yugJWJLY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ZR9zT84IZckgL6VojKqYtTYubTnmKvXzNr8AlFm4gcwHI/sjcPmZzLNtZV53XQyzI 1IqF2do9BVvrI9iRYVZX+unWx1dwf2wJtRRLH8qKOHZxuj7MA7KJ9Hyhkc/Q4Jjd6k ELcF5ywb2uDp7ymvJjK3TnaoLcfG6VA7an7pl+ro= Date: Mon, 16 Dec 2019 10:28:18 +0100 From: Greg Kroah-Hartman To: Andre Tomt Cc: Jack Wang , linux-kernel@vger.kernel.org, stable , Christoph Hellwig , Ming Lei , Jianchao Wang , Jens Axboe , Sasha Levin Subject: Re: [PATCH 4.19 153/306] block: fix the DISCARD request merge (4.19.87+ crash) Message-ID: <20191216092818.GA1203682@kroah.com> References: <20191127203114.766709977@linuxfoundation.org> <20191127203126.845809286@linuxfoundation.org> <4d8343e0-f38a-3e08-edf6-3346b3011ddf@tomt.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4d8343e0-f38a-3e08-edf6-3346b3011ddf@tomt.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Dec 16, 2019 at 10:18:48AM +0100, Andre Tomt wrote: > On 16.12.2019 08:42, Jack Wang wrote: > > Andre Tomt 于2019年12月14日周六 下午3:24写道： > > > > > > 4.19.87, 4.19.88, 4.19.89 all lock up frequently on some of my systems. > > > The same systems run 5.4.3 fine, so the newer trees are probably OK. > > > Reverting this commit on top of 4.19.87 makes everything stable. > > > > > > To trigger it all I have to do is re-rsyncing a directory tree with some > > > changed files churn, it will usually crash in 10 to 30 minutes. > > > > > > The systems crashing has ext4 filesystem on a two ssd md raid1 mounted > > > with the mount option discard. If mounting it without discard, the > > > crashes no longer seem to occur. > > > > > > No oops/panic made it to the ipmi console. I suspect the console is just > > > misbehaving and it didnt really livelock. At one point one line of the > > > crash made it to the console (kernel BUG at block/blk-core.c:1776), and > > > it was enough to pinpoint this commit. Note that the line number might > > > be off, as I was attempting a bisect at the time. > > > > > > This commit also made it to 4.14.x, but I have not tested it. > > Hi Andre, > > > > I noticed one fix is missing for discard merge in 4.19.y > > 2a5cf35cd6c5 ("block: fix single range discard merge") > > > > Can you try if it helps? just "git cherry-pick 2a5cf35cd6c5" > > Indeed, adding this commit on top a clean 4.19.89 fixes the issue. So far > survived about an hour of rsyncing file churn. > Great! Thanks Jack for finding the fix and Andre for reporting this. I'll go queue this fix up right now. greg k-h