From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08097C43381 for ; Tue, 26 Mar 2019 11:42:21 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C794C206DF for ; Tue, 26 Mar 2019 11:42:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="FJbU7yFm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C794C206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:References: To:Subject:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Q1qzbadzBHPVjpv9U4TxCpvoEy02yh9NDmNGv38nJAY=; b=FJbU7yFmGuTw3zlEegQ2wWa1B 6gy07hU6Iivh0ijRMPhVoPfd6HDGVSA8VaxTbSXUxEMADmLW8bDnf92T7hP6iHwMaQ/uWeTvPG/QM 6VZLjPURxTzs5PROJxijTHG2zj/fvnd9BcAkQuaZk70twIda4IWbhBNZR7eWGZslrYb4V8TrajNEY CYbU9m5satNCORkSvVnUhA5ToG0obWtpDHCcRoRS+tHnhd8v7JIEf7/R5rgtnaLgoy4FblxUEdmCc nFvH36yM7UMqPM4pwPWybAxGxD3Kvj2A25B55isSuWjurp6ocE3eIkGoTnZ0DwCRVaTSXpkhDcqXO sJKzqrDjw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h8kTA-0006O8-EI; Tue, 26 Mar 2019 11:42:16 +0000 Received: from szxga06-in.huawei.com ([45.249.212.32] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h8kT5-0006HL-AZ for linux-arm-kernel@lists.infradead.org; Tue, 26 Mar 2019 11:42:14 +0000 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 5BD96F0087E8B8AC2549; Tue, 26 Mar 2019 19:41:55 +0800 (CST) Received: from [127.0.0.1] (10.202.227.238) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.408.0; Tue, 26 Mar 2019 19:41:46 +0800 From: John Garry Subject: Re: [PATCH/RFC] driver core: Postpone DMA tear-down until after devres release To: Geert Uytterhoeven , Greg Kroah-Hartman , Robin Murphy , "Christoph Hellwig" , Marek Szyprowski , "Joerg Roedel" , "Rafael J . Wysocki" References: <20190207193653.18221-1-geert+renesas@glider.be> Message-ID: <9966a420-b081-a8ed-7f7b-20d97c9f7996@huawei.com> Date: Tue, 26 Mar 2019 11:41:38 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20190207193653.18221-1-geert+renesas@glider.be> X-Originating-IP: [10.202.227.238] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190326_044211_953539_EEB5E801 X-CRM114-Status: GOOD ( 15.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: chenxiang , linux-kernel@vger.kernel.org, linux-renesas-soc@vger.kernel.org, iommu@lists.linux-foundation.org, Xiaofei Tan , linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org > > Memory is incorrectly freed using the direct ops, as dma_map_ops = NULL. > Oops... > > After reversing the order of the calls to arch_teardown_dma_ops() and > devres_release_all(), dma_map_ops is still valid, and the DMA memory is > now released using __iommu_free_attrs(): > > +sata_rcar ee300000.sata: dmam_release:32: size 2048 vaddr ffffff8012145000 dma_handle 0x0x00000000fffff000 attrs 0x0 > +sata_rcar ee300000.sata: dma_free_attrs:289: size 2048, ops = iommu_dma_ops > +sata_rcar ee300000.sata: dma_free_attrs:311: calling __iommu_free_attrs() > --- > drivers/base/dd.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/base/dd.c b/drivers/base/dd.c > index 8ac10af17c0043a3..d62487d024559620 100644 > --- a/drivers/base/dd.c > +++ b/drivers/base/dd.c > @@ -968,9 +968,9 @@ static void __device_release_driver(struct device *dev, struct device *parent) > drv->remove(dev); > > device_links_driver_cleanup(dev); > - arch_teardown_dma_ops(dev); > > devres_release_all(dev); > + arch_teardown_dma_ops(dev); > dev->driver = NULL; Hi guys, Could there still be the same problem in the error path of really_probe(): static int really_probe(struct device *dev, struct device_driver *drv) { [...] goto done; probe_failed: arch_teardown_dma_ops(dev); dma_failed: if (dev->bus) blocking_notifier_call_chain(&dev->bus->p->bus_notifier, BUS_NOTIFY_DRIVER_NOT_BOUND, dev); pinctrl_bind_failed: device_links_no_driver(dev); devres_release_all(dev); driver_sysfs_remove(dev); dev->driver = NULL; dev_set_drvdata(dev, NULL); We seem to be able to call arch_teardown_dma_ops() prior to devres_release_all() if we reach probe_failed label. We have seen this crash when our driver probe fails on a dev branch based on v5.1-rc1: [ 87.896707] hisi_sas_v3_hw 0000:74:02.0: Adding to iommu group 2 [ 87.909765] scsi host1: hisi_sas_v3_hw [ 89.127958] hisi_sas_v3_hw 0000:74:02.0: evaluate _DSM failed [ 89.134043] BUG: Bad page state in process swapper/0 pfn:313f5 [ 89.139965] page:ffff7e0000c4fd40 count:1 mapcount:0 mapping:0000000000000000 index:0x0 [ 89.147960] flags: 0xfffe00000001000(reserved) [ 89.152398] raw: 0fffe00000001000 ffff7e0000c4fd48 ffff7e0000c4fd48 0000000000000000 [ 89.160130] raw: 0000000000000000 0000000000000000 00000001ffffffff 0000000000000000 [ 89.167861] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set [ 89.174290] bad because of flags: 0x1000(reserved) [ 89.179070] Modules linked in: [ 89.182117] CPU: 49 PID: 1 Comm: swapper/0 Not tainted 5.1.0-rc1-43081-g22d97fd-dirty #1433 [ 89.190453] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI RC0 - V1.12.01 01/29/2019 [ 89.198876] Call trace: [ 89.201316] dump_backtrace+0x0/0x118 [ 89.204965] show_stack+0x14/0x1c [ 89.208272] dump_stack+0xa4/0xc8 [ 89.211576] bad_page+0xe4/0x13c [ 89.214791] free_pages_check_bad+0x4c/0xc0 [ 89.218961] __free_pages_ok+0x30c/0x340 [ 89.222871] __free_pages+0x30/0x44 [ 89.226347] __dma_direct_free_pages+0x30/0x38 [ 89.230777] dma_direct_free+0x24/0x38 [ 89.234513] dma_free_attrs+0x9c/0xd8 [ 89.238161] dmam_release+0x20/0x28 [ 89.241640] release_nodes+0x17c/0x220 [ 89.245375] devres_release_all+0x34/0x54 [ 89.249371] really_probe+0xc4/0x2c8 [ 89.252933] driver_probe_device+0x58/0xfc [ 89.257016] device_driver_attach+0x68/0x70 [ 89.261185] __driver_attach+0x94/0xdc [ 89.264921] bus_for_each_dev+0x5c/0xb4 [ 89.268744] driver_attach+0x20/0x28 [ 89.272306] bus_add_driver+0x14c/0x200 [ 89.276128] driver_register+0x6c/0x124 [ 89.279953] __pci_register_driver+0x48/0x50 [ 89.284213] sas_v3_pci_driver_init+0x20/0x28 [ 89.288557] do_one_initcall+0x40/0x25c [ 89.292381] kernel_init_freeable+0x2b8/0x3c0 [ 89.296727] kernel_init+0x10/0x100 [ 89.300202] ret_from_fork+0x10/0x18 [ 89.303773] Disabling lock debugging due to kernel taint [ 89.309076] BUG: Bad page state in process swapper/0 pfn:313f6 [ 89.314988] page:ffff7e0000c4fd80 count:1 mapcount:0 mapping:0000000000000000 index:0x0 [ 89.322983] flags: 0xfffe00000001000(reserved) [ 89.327417] raw: 0fffe00000001000 ffff7e0000c4fd88 ffff7e0000c4fd88 0000000000000000 [ 89.335149] raw: 0000000000000000 0000000000000000 00000001ffffffff 0000000000000000 Thanks, John > dev_set_drvdata(dev, NULL); > if (dev->pm_domain && dev->pm_domain->dismiss) > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel