From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCD23C433E6 for ; Tue, 2 Feb 2021 00:27:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B502764E8D for ; Tue, 2 Feb 2021 00:27:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231339AbhBBA1B (ORCPT ); Mon, 1 Feb 2021 19:27:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231299AbhBBA07 (ORCPT ); Mon, 1 Feb 2021 19:26:59 -0500 Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A254C061793 for ; Mon, 1 Feb 2021 16:26:09 -0800 (PST) Received: by mail-ot1-x32d.google.com with SMTP id n42so18194581ota.12 for ; Mon, 01 Feb 2021 16:26:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZaKN0kS6u64uQV+6FGLHQ/HFDrwuOhBJX+D4+gTt6r0=; b=tCDZIyKRfYkflVO/mtEEY/OhC27bA9kqcX/5z+t4BtDBqfjp5sZ49VjvxVeoplOarj AHmY7N+os6HUVETD7QEgDovkqXB9vRw9mosvTMB0EzWzWlaO05AsR204OzZKZglBwYiX pHQBMKa7/HwClvsBCbv5i43x7LoXh+9DL2zfBgKB+ONF2UqWACyeWSzlt2oyy3ElQkYU jU42fR2o3wK8iolXAqeuVQN1utwYgYnSs8VYPrrMqiVJVYcqHdUGJWMc9C9X41IuMgq9 H5UcsTgNmbqCwr1LI57T+EnrptCtYRBZDYWAvgWbsMxvK6h1QSipoNDTD9tPto4A4sar N7Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZaKN0kS6u64uQV+6FGLHQ/HFDrwuOhBJX+D4+gTt6r0=; b=qJ3rwDrWLvs1u0yRKUCjdfRdM//rq2OUaPj8y6ynSMw7O+7yuX2Jy9nvLdItw3qwY3 6YTBWWOd/1ibD2CBVTtKUa52DG6DeZpyXyp8bpkOj4/ap/elZTPvfkadLolMY+rljSCK WSOu4RJDPLmG2WX7vPES1YCQpH9wGjVI/LVYk6HPbsOygHpVHuqxGznGYCWc7993pxED r3RL78mn6GwBf8a09J1BOU6ScXeTfGP5U/m64kfQVDtcxrmBNQyA5tn/HsX9y2XbTORw IbVXJ7duzuOzk7N1HoyUkA3f+BixC4sXnT4nfc24n+Px8wt0nR6dRJaHvSGpxSPCXV7t T4nQ== X-Gm-Message-State: AOAM533eDF/olvH0XoUabxNNP6ELsD5co0ZBxXpx888gn+6X/tfVdHpW hRydVxsh9Ykc8RxxdJVb0KNB1381wJtN6RDU51YS4Q== X-Google-Smtp-Source: ABdhPJyFgXBxhpe6iO2OHTC17kqmn9HUrbruwF7fouFTXMMPS3G9sjeY2vc7eAvB7AwHVf2zMFQpAJVvevVDT4EA30k= X-Received: by 2002:a9d:7d12:: with SMTP id v18mr13319071otn.205.1612225568439; Mon, 01 Feb 2021 16:26:08 -0800 (PST) MIME-Version: 1.0 References: <20210201183017.3339130-1-jxgao@google.com> <20210201183017.3339130-4-jxgao@google.com> <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com> In-Reply-To: From: Jianxiong Gao Date: Mon, 1 Feb 2021 16:25:55 -0800 Message-ID: Subject: Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver. To: Keith Busch Cc: Erdem Aktas , Marc Orr , Christoph Hellwig , m.szyprowski@samsung.com, Robin Murphy , gregkh@linuxfoundation.org, Saravana Kannan , heikki.krogerus@linux.intel.com, rafael.j.wysocki@intel.com, Andy Shevchenko , dan.j.williams@intel.com, bgolaszewski@baylibre.com, jroedel@suse.de, iommu@lists.linux-foundation.org, Konrad Rzeszutek Wilk , axboe@fb.com, sagi@grimberg.me, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > Why is this setting being done and undone on each IO? Wouldn't it be > more efficient to set it once during device initialization? I agree that setting it once is the right way of doing it. So I have changed the patch to enable the mask once in nvme_probe. drivers/nvme/host/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 81e6389b2042..4ce78373f98d 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work) */ dma_set_max_seg_size(dev->dev, 0xffffffff); + if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1)) + dev_warn(dev->dev, "dma_set_min_align_mask failed to set offset\n"); + mutex_unlock(&dev->shutdown_lock); /* However on boot of the system, the following error happens occasionally. The error seems related to Journal service. Whenever "Stopping Journal Service..." appears, the boot succeeds. Otherwise boot fails with the following message. ----------------------------log start here-------------------------- [ OK ] Started Journal Service. [ 10.774545] xfs filesystem being remounted at / supports timestamps until 2038 (0x7fffffff) [ OK ] Started Remount Root and Kernel File Systems. Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Create Static [ 10.804340] systemd-journald[780]: Received request to flush runtime journal from PID 1 Device Nodes in /dev... [ OK ] Started Load/Save Random Seed. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Starting File System Check on /dev/disk/by-uuid/7281-17FC... [ OK ] Started File System Check on /dev/disk/by-uuid/7281-17FC. Mounting /boot/efi... [ OK ] Mounted /boo[ 11.203461] systemd[1]: segfault at 2e0 ip 000055b08607cc24 sp 00007ffe13809090 error 4 in systemd[55b086000000+140000] t/efi. [ 11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d 6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48 8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48 89 de ---------------log ends here----------- > Based on the Keith's comment it needs to be completely avoided in hot path. > > Did you get a chance to bisect the problem in the rc6 ? We need to know the > root cause otherwise it might happen again after we add this patch. I am now trying to bisect the problem. I am not sure how the mapping offset could have caused the error. Any suggestions are appreciated. -- Jianxiong Gao From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B78AC433DB for ; Tue, 2 Feb 2021 00:26:19 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 435BB64E2A for ; Tue, 2 Feb 2021 00:26:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 435BB64E2A Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Jhp/pWFHYmk/uXJ+MpVvBIqaYz18Iy9wLRiPIwA+F2Q=; b=nQButfSlvoQnvbUWqyJlAxu8l NfbMr2/Qasywq4QuoVHEqcN9+uLZ0Ne6cp27z6eh8wSm4T8hPEofdhaTbaK9G72nFhhLoOyyzslue VkwUTCn6mdnhLbOnLAr4y3g1XI2zds2GJ0DsKEiYPtHSvS57Yn25PZzGDR0B4m55rm5NOgkQ6greK 6Y0NrW3VNbVnb+ku5AKXUNXlEcCU1GEm9XT41Z9uVTMvcVGQ5vkCuLEiXYsWgmJ9fh6eGNtZ4GJLQ c+95zE8gLep+Gb5XmQQlfcCkFMEZyTStyk8PG6zxNSkqLiD28n9heJkoJlx34DZrHB6aGOUz7MiBj 90Z4hCSHQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6jWI-000732-Kb; Tue, 02 Feb 2021 00:26:14 +0000 Received: from mail-ot1-x334.google.com ([2607:f8b0:4864:20::334]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6jWE-00072Y-L4 for linux-nvme@lists.infradead.org; Tue, 02 Feb 2021 00:26:12 +0000 Received: by mail-ot1-x334.google.com with SMTP id h14so18232121otr.4 for ; Mon, 01 Feb 2021 16:26:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZaKN0kS6u64uQV+6FGLHQ/HFDrwuOhBJX+D4+gTt6r0=; b=tCDZIyKRfYkflVO/mtEEY/OhC27bA9kqcX/5z+t4BtDBqfjp5sZ49VjvxVeoplOarj AHmY7N+os6HUVETD7QEgDovkqXB9vRw9mosvTMB0EzWzWlaO05AsR204OzZKZglBwYiX pHQBMKa7/HwClvsBCbv5i43x7LoXh+9DL2zfBgKB+ONF2UqWACyeWSzlt2oyy3ElQkYU jU42fR2o3wK8iolXAqeuVQN1utwYgYnSs8VYPrrMqiVJVYcqHdUGJWMc9C9X41IuMgq9 H5UcsTgNmbqCwr1LI57T+EnrptCtYRBZDYWAvgWbsMxvK6h1QSipoNDTD9tPto4A4sar N7Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZaKN0kS6u64uQV+6FGLHQ/HFDrwuOhBJX+D4+gTt6r0=; b=D9KHT8K3K1F1wLzH6TaPbGZSoTkae59VtF+aRBCXNc7ZM1//sbtplUr8SGN+yZzYM0 xlgJ9TgBHWZNLvYPlXBDkOX85NmbO7a1vbUkr7oq8wjwlXtjiQeKMxyVmNRvIK8iBBBp 4oOMU0w3PQhKO0ofEbuEbee6gr7enj/Z+MXATeTCae1My0hCpDJ5nRkbM8E08CKE3o3h N8/qqsu4MRI6wjg4v9HJp2DpagiA1YS4lpJ7dIAJFF7sGEk95aJZCFiiLMD1CMIQaweg jIiENAEl8VWT/DkRlcD5h4MuP5zJHkhEu5gzpyR/cbXlHhw+5UTr/JOobvCh0RsC0QkU bGHw== X-Gm-Message-State: AOAM530XSRp9DqoygbZHRa+6wVdm6SBce/66EvIbjdMxEwv9BQXujuQX fBqe7jgtHXqytAyqbQHXeB4UiZlJbALawyX7v3ka9Q== X-Google-Smtp-Source: ABdhPJyFgXBxhpe6iO2OHTC17kqmn9HUrbruwF7fouFTXMMPS3G9sjeY2vc7eAvB7AwHVf2zMFQpAJVvevVDT4EA30k= X-Received: by 2002:a9d:7d12:: with SMTP id v18mr13319071otn.205.1612225568439; Mon, 01 Feb 2021 16:26:08 -0800 (PST) MIME-Version: 1.0 References: <20210201183017.3339130-1-jxgao@google.com> <20210201183017.3339130-4-jxgao@google.com> <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com> In-Reply-To: From: Jianxiong Gao Date: Mon, 1 Feb 2021 16:25:55 -0800 Message-ID: Subject: Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver. To: Keith Busch X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210201_192610_746474_0C68EBDE X-CRM114-Status: GOOD ( 21.90 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heikki.krogerus@linux.intel.com, sagi@grimberg.me, Saravana Kannan , Konrad Rzeszutek Wilk , Marc Orr , gregkh@linuxfoundation.org, rafael.j.wysocki@intel.com, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, axboe@fb.com, bgolaszewski@baylibre.com, iommu@lists.linux-foundation.org, jroedel@suse.de, Erdem Aktas , dan.j.williams@intel.com, Andy Shevchenko , Robin Murphy , Christoph Hellwig , m.szyprowski@samsung.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org > Why is this setting being done and undone on each IO? Wouldn't it be > more efficient to set it once during device initialization? I agree that setting it once is the right way of doing it. So I have changed the patch to enable the mask once in nvme_probe. drivers/nvme/host/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 81e6389b2042..4ce78373f98d 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work) */ dma_set_max_seg_size(dev->dev, 0xffffffff); + if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1)) + dev_warn(dev->dev, "dma_set_min_align_mask failed to set offset\n"); + mutex_unlock(&dev->shutdown_lock); /* However on boot of the system, the following error happens occasionally. The error seems related to Journal service. Whenever "Stopping Journal Service..." appears, the boot succeeds. Otherwise boot fails with the following message. ----------------------------log start here-------------------------- [ OK ] Started Journal Service. [ 10.774545] xfs filesystem being remounted at / supports timestamps until 2038 (0x7fffffff) [ OK ] Started Remount Root and Kernel File Systems. Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Create Static [ 10.804340] systemd-journald[780]: Received request to flush runtime journal from PID 1 Device Nodes in /dev... [ OK ] Started Load/Save Random Seed. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Starting File System Check on /dev/disk/by-uuid/7281-17FC... [ OK ] Started File System Check on /dev/disk/by-uuid/7281-17FC. Mounting /boot/efi... [ OK ] Mounted /boo[ 11.203461] systemd[1]: segfault at 2e0 ip 000055b08607cc24 sp 00007ffe13809090 error 4 in systemd[55b086000000+140000] t/efi. [ 11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d 6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48 8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48 89 de ---------------log ends here----------- > Based on the Keith's comment it needs to be completely avoided in hot path. > > Did you get a chance to bisect the problem in the rc6 ? We need to know the > root cause otherwise it might happen again after we add this patch. I am now trying to bisect the problem. I am not sure how the mapping offset could have caused the error. Any suggestions are appreciated. -- Jianxiong Gao _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E47DCC433E0 for ; Tue, 2 Feb 2021 00:26:13 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6002564EB9 for ; Tue, 2 Feb 2021 00:26:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6002564EB9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lists.linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 121F32153E; Tue, 2 Feb 2021 00:26:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B8of1Tsb44vw; Tue, 2 Feb 2021 00:26:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 0740120334; Tue, 2 Feb 2021 00:26:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EC5FBC0174; Tue, 2 Feb 2021 00:26:10 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 83E3FC013A for ; Tue, 2 Feb 2021 00:26:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 6389021509 for ; Tue, 2 Feb 2021 00:26:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kiVbvTK3x4Lc for ; Tue, 2 Feb 2021 00:26:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by silver.osuosl.org (Postfix) with ESMTPS id 706AF20334 for ; Tue, 2 Feb 2021 00:26:09 +0000 (UTC) Received: by mail-ot1-f54.google.com with SMTP id h14so18232120otr.4 for ; Mon, 01 Feb 2021 16:26:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZaKN0kS6u64uQV+6FGLHQ/HFDrwuOhBJX+D4+gTt6r0=; b=tCDZIyKRfYkflVO/mtEEY/OhC27bA9kqcX/5z+t4BtDBqfjp5sZ49VjvxVeoplOarj AHmY7N+os6HUVETD7QEgDovkqXB9vRw9mosvTMB0EzWzWlaO05AsR204OzZKZglBwYiX pHQBMKa7/HwClvsBCbv5i43x7LoXh+9DL2zfBgKB+ONF2UqWACyeWSzlt2oyy3ElQkYU jU42fR2o3wK8iolXAqeuVQN1utwYgYnSs8VYPrrMqiVJVYcqHdUGJWMc9C9X41IuMgq9 H5UcsTgNmbqCwr1LI57T+EnrptCtYRBZDYWAvgWbsMxvK6h1QSipoNDTD9tPto4A4sar N7Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZaKN0kS6u64uQV+6FGLHQ/HFDrwuOhBJX+D4+gTt6r0=; b=qIdxdU+vbGO5l7LOmefwgikrDI8tb0eJyTwuTYr3ChBt1aRV8RsylHhZo/xVG25xkm R6K+f5l1pPdyBQHgwI4Up9DRqHXaaaiUhc0B10mkTuUU06b4IeJJOwd/26GvQq0tohmm RFiGOjWwCdYAoeUqYQSPM6i99Qfyqn1Od3IQifvPknSkCwaK37hJaUimlo9mdh6iSmuc XQy00fS6ttX4dMETUgI48B91AAWGmFw4YWL2iC/TdWQqCRn4KLz7/EYyrGAw0mhZnxQP kJ33rf2SgJFRAEwvVzziAlzQE7+Q3fhpijRKoz5p6fS71Ou/1zuTm1UHaZEoSGG4V4hu mrEg== X-Gm-Message-State: AOAM530PqNM0AIigmEDyv7of0u43dxyxxOYxRT3+PdHmF8edpWoHOfEk 8nd5CpuJr5EZxE5mycvSwMzFk6S/hN3ADWQ7SUK5TQ== X-Google-Smtp-Source: ABdhPJyFgXBxhpe6iO2OHTC17kqmn9HUrbruwF7fouFTXMMPS3G9sjeY2vc7eAvB7AwHVf2zMFQpAJVvevVDT4EA30k= X-Received: by 2002:a9d:7d12:: with SMTP id v18mr13319071otn.205.1612225568439; Mon, 01 Feb 2021 16:26:08 -0800 (PST) MIME-Version: 1.0 References: <20210201183017.3339130-1-jxgao@google.com> <20210201183017.3339130-4-jxgao@google.com> <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com> In-Reply-To: Date: Mon, 1 Feb 2021 16:25:55 -0800 Message-ID: Subject: Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver. To: Keith Busch Cc: heikki.krogerus@linux.intel.com, sagi@grimberg.me, Saravana Kannan , Konrad Rzeszutek Wilk , Marc Orr , gregkh@linuxfoundation.org, rafael.j.wysocki@intel.com, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, axboe@fb.com, bgolaszewski@baylibre.com, iommu@lists.linux-foundation.org, jroedel@suse.de, dan.j.williams@intel.com, Andy Shevchenko , Robin Murphy , Christoph Hellwig X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Jianxiong Gao via iommu Reply-To: Jianxiong Gao Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" > Why is this setting being done and undone on each IO? Wouldn't it be > more efficient to set it once during device initialization? I agree that setting it once is the right way of doing it. So I have changed the patch to enable the mask once in nvme_probe. drivers/nvme/host/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 81e6389b2042..4ce78373f98d 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work) */ dma_set_max_seg_size(dev->dev, 0xffffffff); + if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1)) + dev_warn(dev->dev, "dma_set_min_align_mask failed to set offset\n"); + mutex_unlock(&dev->shutdown_lock); /* However on boot of the system, the following error happens occasionally. The error seems related to Journal service. Whenever "Stopping Journal Service..." appears, the boot succeeds. Otherwise boot fails with the following message. ----------------------------log start here-------------------------- [ OK ] Started Journal Service. [ 10.774545] xfs filesystem being remounted at / supports timestamps until 2038 (0x7fffffff) [ OK ] Started Remount Root and Kernel File Systems. Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Create Static [ 10.804340] systemd-journald[780]: Received request to flush runtime journal from PID 1 Device Nodes in /dev... [ OK ] Started Load/Save Random Seed. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Starting File System Check on /dev/disk/by-uuid/7281-17FC... [ OK ] Started File System Check on /dev/disk/by-uuid/7281-17FC. Mounting /boot/efi... [ OK ] Mounted /boo[ 11.203461] systemd[1]: segfault at 2e0 ip 000055b08607cc24 sp 00007ffe13809090 error 4 in systemd[55b086000000+140000] t/efi. [ 11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d 6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48 8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48 89 de ---------------log ends here----------- > Based on the Keith's comment it needs to be completely avoided in hot path. > > Did you get a chance to bisect the problem in the rc6 ? We need to know the > root cause otherwise it might happen again after we add this patch. I am now trying to bisect the problem. I am not sure how the mapping offset could have caused the error. Any suggestions are appreciated. -- Jianxiong Gao _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu