All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jianxiong Gao <jxgao@google.com>
To: Keith Busch <kbusch@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>,
	Marc Orr <marcorr@google.com>, Christoph Hellwig <hch@lst.de>,
	m.szyprowski@samsung.com, Robin Murphy <robin.murphy@arm.com>,
	gregkh@linuxfoundation.org,
	Saravana Kannan <saravanak@google.com>,
	heikki.krogerus@linux.intel.com, rafael.j.wysocki@intel.com,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	dan.j.williams@intel.com, bgolaszewski@baylibre.com,
	jroedel@suse.de, iommu@lists.linux-foundation.org,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	axboe@fb.com, sagi@grimberg.me, linux-nvme@lists.infradead.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.
Date: Mon, 1 Feb 2021 16:25:55 -0800	[thread overview]
Message-ID: <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com> (raw)
In-Reply-To: <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>

> Why is this setting being done and undone on each IO? Wouldn't it be
> more efficient to set it once during device initialization?

I agree that setting it once is the right way of doing it.

So I have changed the patch to enable the mask once in nvme_probe.

 drivers/nvme/host/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..4ce78373f98d 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work)
         */
        dma_set_max_seg_size(dev->dev, 0xffffffff);

+       if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))
+               dev_warn(dev->dev, "dma_set_min_align_mask failed to
set offset\n");
+
        mutex_unlock(&dev->shutdown_lock);

        /*

However on boot of the system, the following error happens occasionally.
The error seems related to Journal service. Whenever "Stopping Journal
Service..."
appears, the boot succeeds. Otherwise boot fails with the following message.

----------------------------log start here--------------------------
[  OK  ] Started Journal Service.
[   10.774545] xfs filesystem being remounted at / supports timestamps
until 2038 (0x7fffffff)
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Flush Journal to Persistent Storage...
         Starting Load/Save Random Seed...
         Starting Create Static [   10.804340] systemd-journald[780]:
Received request to flush runtime journal from PID 1
Device Nodes in /dev...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Flush Journal to Persistent Storage.
[  OK  ] Started Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Reached target Local File Systems (Pre).
         Starting File System Check on /dev/disk/by-uuid/7281-17FC...
[  OK  ] Started File System Check on /dev/disk/by-uuid/7281-17FC.
         Mounting /boot/efi...
[  OK  ] Mounted /boo[   11.203461] systemd[1]: segfault at 2e0 ip
000055b08607cc24 sp 00007ffe13809090 error 4 in
systemd[55b086000000+140000]
t/efi.
[   11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d
6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48
8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48
89 de
---------------log ends here-----------

> Based on the Keith's comment it needs to be completely avoided in hot path.
>
> Did you get a chance to bisect the problem in the rc6 ? We need to know the
> root cause otherwise it might happen again after we add this patch.

I am now trying to bisect the problem.
I am not sure how the mapping offset could have caused the error.
Any suggestions are appreciated.

--
Jianxiong Gao

WARNING: multiple messages have this Message-ID (diff)
From: Jianxiong Gao <jxgao@google.com>
To: Keith Busch <kbusch@kernel.org>
Cc: heikki.krogerus@linux.intel.com, sagi@grimberg.me,
	Saravana Kannan <saravanak@google.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Marc Orr <marcorr@google.com>,
	gregkh@linuxfoundation.org, rafael.j.wysocki@intel.com,
	linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org,
	axboe@fb.com, bgolaszewski@baylibre.com,
	iommu@lists.linux-foundation.org, jroedel@suse.de,
	Erdem Aktas <erdemaktas@google.com>,
	dan.j.williams@intel.com,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Christoph Hellwig <hch@lst.de>,
	m.szyprowski@samsung.com
Subject: Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.
Date: Mon, 1 Feb 2021 16:25:55 -0800	[thread overview]
Message-ID: <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com> (raw)
In-Reply-To: <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>

> Why is this setting being done and undone on each IO? Wouldn't it be
> more efficient to set it once during device initialization?

I agree that setting it once is the right way of doing it.

So I have changed the patch to enable the mask once in nvme_probe.

 drivers/nvme/host/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..4ce78373f98d 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work)
         */
        dma_set_max_seg_size(dev->dev, 0xffffffff);

+       if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))
+               dev_warn(dev->dev, "dma_set_min_align_mask failed to
set offset\n");
+
        mutex_unlock(&dev->shutdown_lock);

        /*

However on boot of the system, the following error happens occasionally.
The error seems related to Journal service. Whenever "Stopping Journal
Service..."
appears, the boot succeeds. Otherwise boot fails with the following message.

----------------------------log start here--------------------------
[  OK  ] Started Journal Service.
[   10.774545] xfs filesystem being remounted at / supports timestamps
until 2038 (0x7fffffff)
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Flush Journal to Persistent Storage...
         Starting Load/Save Random Seed...
         Starting Create Static [   10.804340] systemd-journald[780]:
Received request to flush runtime journal from PID 1
Device Nodes in /dev...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Flush Journal to Persistent Storage.
[  OK  ] Started Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Reached target Local File Systems (Pre).
         Starting File System Check on /dev/disk/by-uuid/7281-17FC...
[  OK  ] Started File System Check on /dev/disk/by-uuid/7281-17FC.
         Mounting /boot/efi...
[  OK  ] Mounted /boo[   11.203461] systemd[1]: segfault at 2e0 ip
000055b08607cc24 sp 00007ffe13809090 error 4 in
systemd[55b086000000+140000]
t/efi.
[   11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d
6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48
8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48
89 de
---------------log ends here-----------

> Based on the Keith's comment it needs to be completely avoided in hot path.
>
> Did you get a chance to bisect the problem in the rc6 ? We need to know the
> root cause otherwise it might happen again after we add this patch.

I am now trying to bisect the problem.
I am not sure how the mapping offset could have caused the error.
Any suggestions are appreciated.

--
Jianxiong Gao

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

WARNING: multiple messages have this Message-ID (diff)
From: Jianxiong Gao via iommu <iommu@lists.linux-foundation.org>
To: Keith Busch <kbusch@kernel.org>
Cc: heikki.krogerus@linux.intel.com, sagi@grimberg.me,
	Saravana Kannan <saravanak@google.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Marc Orr <marcorr@google.com>,
	gregkh@linuxfoundation.org, rafael.j.wysocki@intel.com,
	linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org,
	axboe@fb.com, bgolaszewski@baylibre.com,
	iommu@lists.linux-foundation.org, jroedel@suse.de,
	dan.j.williams@intel.com,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.
Date: Mon, 1 Feb 2021 16:25:55 -0800	[thread overview]
Message-ID: <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com> (raw)
In-Reply-To: <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>

> Why is this setting being done and undone on each IO? Wouldn't it be
> more efficient to set it once during device initialization?

I agree that setting it once is the right way of doing it.

So I have changed the patch to enable the mask once in nvme_probe.

 drivers/nvme/host/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..4ce78373f98d 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work)
         */
        dma_set_max_seg_size(dev->dev, 0xffffffff);

+       if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))
+               dev_warn(dev->dev, "dma_set_min_align_mask failed to
set offset\n");
+
        mutex_unlock(&dev->shutdown_lock);

        /*

However on boot of the system, the following error happens occasionally.
The error seems related to Journal service. Whenever "Stopping Journal
Service..."
appears, the boot succeeds. Otherwise boot fails with the following message.

----------------------------log start here--------------------------
[  OK  ] Started Journal Service.
[   10.774545] xfs filesystem being remounted at / supports timestamps
until 2038 (0x7fffffff)
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Flush Journal to Persistent Storage...
         Starting Load/Save Random Seed...
         Starting Create Static [   10.804340] systemd-journald[780]:
Received request to flush runtime journal from PID 1
Device Nodes in /dev...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Flush Journal to Persistent Storage.
[  OK  ] Started Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Reached target Local File Systems (Pre).
         Starting File System Check on /dev/disk/by-uuid/7281-17FC...
[  OK  ] Started File System Check on /dev/disk/by-uuid/7281-17FC.
         Mounting /boot/efi...
[  OK  ] Mounted /boo[   11.203461] systemd[1]: segfault at 2e0 ip
000055b08607cc24 sp 00007ffe13809090 error 4 in
systemd[55b086000000+140000]
t/efi.
[   11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d
6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48
8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48
89 de
---------------log ends here-----------

> Based on the Keith's comment it needs to be completely avoided in hot path.
>
> Did you get a chance to bisect the problem in the rc6 ? We need to know the
> root cause otherwise it might happen again after we add this patch.

I am now trying to bisect the problem.
I am not sure how the mapping offset could have caused the error.
Any suggestions are appreciated.

--
Jianxiong Gao
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  parent reply	other threads:[~2021-02-02  0:27 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-01 18:30 [PATCH V2 0/3] SWIOTLB: Preserve swiotlb map offset when needed Jianxiong Gao
2021-02-01 18:30 ` Jianxiong Gao via iommu
2021-02-01 18:30 ` Jianxiong Gao
2021-02-01 18:30 ` [PATCH V2 1/3] Adding page_offset_mask to device_dma_parameters Jianxiong Gao
2021-02-01 18:30   ` Jianxiong Gao via iommu
2021-02-01 18:30   ` Jianxiong Gao
2021-02-01 18:30 ` [PATCH V2 2/3] Add swiotlb offset preserving mapping when dma_dma_parameters->page_offset_mask is non zero Jianxiong Gao
2021-02-01 18:30   ` Jianxiong Gao via iommu
2021-02-01 18:30   ` Jianxiong Gao
2021-02-01 18:30 ` [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver Jianxiong Gao
2021-02-01 18:30   ` Jianxiong Gao via iommu
2021-02-01 18:30   ` Jianxiong Gao
2021-02-01 18:55   ` Andy Shevchenko
2021-02-01 18:55     ` Andy Shevchenko
2021-02-01 18:55     ` Andy Shevchenko
2021-02-01 19:35     ` Jianxiong Gao
2021-02-01 19:35       ` Jianxiong Gao via iommu
2021-02-01 19:35       ` Jianxiong Gao
2021-02-01 20:57   ` Keith Busch
2021-02-01 20:57     ` Keith Busch
2021-02-01 20:57     ` Keith Busch
2021-02-01 21:16     ` Jianxiong Gao via iommu
2021-02-01 21:22       ` Jianxiong Gao
2021-02-01 21:22         ` Jianxiong Gao via iommu
2021-02-01 21:22         ` Jianxiong Gao
2021-02-01 23:59         ` Chaitanya Kulkarni
2021-02-01 23:59           ` Chaitanya Kulkarni
2021-02-01 23:59           ` Chaitanya Kulkarni
2021-02-02  0:25         ` Jianxiong Gao [this message]
2021-02-02  0:25           ` Jianxiong Gao via iommu
2021-02-02  0:25           ` Jianxiong Gao
2021-02-02 11:21           ` Andy Shevchenko
2021-02-02 11:21             ` Andy Shevchenko
2021-02-02 11:21             ` Andy Shevchenko
2021-02-02 12:07             ` Robin Murphy
2021-02-02 12:07               ` Robin Murphy
2021-02-02 12:07               ` Robin Murphy
2021-02-03 13:37         ` Christoph Hellwig
2021-02-03 13:37           ` Christoph Hellwig
2021-02-03 13:37           ` Christoph Hellwig
2021-02-03 16:47           ` Jianxiong Gao
2021-02-03 16:47             ` Jianxiong Gao via iommu
2021-02-03 16:47             ` Jianxiong Gao
2021-02-02 11:53 ` [PATCH V2 0/3] SWIOTLB: Preserve swiotlb map offset when needed Greg KH
2021-02-02 11:53   ` Greg KH
2021-02-02 11:53   ` Greg KH

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com \
    --to=jxgao@google.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=axboe@fb.com \
    --cc=bgolaszewski@baylibre.com \
    --cc=dan.j.williams@intel.com \
    --cc=erdemaktas@google.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hch@lst.de \
    --cc=heikki.krogerus@linux.intel.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jroedel@suse.de \
    --cc=kbusch@kernel.org \
    --cc=konrad.wilk@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=m.szyprowski@samsung.com \
    --cc=marcorr@google.com \
    --cc=rafael.j.wysocki@intel.com \
    --cc=robin.murphy@arm.com \
    --cc=sagi@grimberg.me \
    --cc=saravanak@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.