All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Garry <john.garry@huawei.com>
To: Christoph Hellwig <hch@lst.de>
Cc: sagi@grimberg.me, Will Deacon <will@kernel.org>,
	linux-nvme@lists.infradead.org, axboe@fb.com,
	Keith Busch <kbusch@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Alexey Dobriyan <adobriyan@gmail.com>
Subject: Re: [PATCH] nvme-pci: slimmer CQ head update
Date: Wed, 6 May 2020 17:26:35 +0100	[thread overview]
Message-ID: <4155a814-798c-0c7e-5433-daf719c0345c@huawei.com> (raw)
In-Reply-To: <20200506143519.GA570@lst.de>

+ arm64 guys (Please note WARN below, generated when testing NVMe)

On 06/05/2020 15:35, Christoph Hellwig wrote:> On Wed, May 06, 2020 at 
02:44:50PM +0100, John Garry wrote:
>> I'd rather hear the maintainer’s opinion before bothering testing this...
> 
> As the other maintainer - please give it a spin. 

ok, so I have tested with the modification from Keith (to avoid the 
READ_ONCE()), and it's ok for use_threaded_interrupts=1.

However, for use_threaded_interrupts=0, I see a new issue:

[  122.524290] WARNING: CPU: 86 PID: 1157 at 
drivers/iommu/io-pgtable-arm.c:304
__arm_lpae_map+0x1d0/0x2bc
[R(60)] [0.0% d[  122.533640] Modules linked in:
[  122.538062] CPU: 86 PID: 1157 Comm: fio Tainted: G        W 
5.7.0-rc4
-ga3c4a5a-dirty #140
[  122.546892] Hardware name: Huawei TaiShan 2280 V2/BC82AMDC, BIOS 
2280-V2 CS V
3.B220.02 03/27/2020
one] [4822MB/0KB[  122.555722] pstate: 60400009 (nZCv daif +PAN -UAO)
[  122.561870] pc : __arm_lpae_map+0x1d0/0x2bc
[  122.566036] lr : __arm_lpae_map+0xf0/0x2bc
/0KB /s] [1235K/[  122.570114] sp : ffff800024b5b4b0
[  122.574794] x29: ffff800024b5b4b0 x28: ffffb62a9e503d4c
0/0 iops] [eta 1[  122.580083] x27: 0000000000001000 x26: 0000000000000001
[  122.586750] x25: ffff2027c6e10980 x24: 0000000000000f44
158050441d:06h:5[  122.592038] x23: 00000027d10f9000 x22: 00000000ef130000
[  122.598706] x21: 0000000000001000 x20: 0000000000000980
[  122.603994] x19: ffff0027dba3e200 x18: 0000000000000000
[  122.609970] x17: 0000000000000000 x16: 0000000000000000
[  122.615257] x15: 0000000000000000 x14: 0000000000000000
[  122.620544] x13: 0000000000000000 x12: 0000000000000000
[  122.625831] x11: 0000000000000002 x10: 0000000000001000
[  122.631119] x9 : 0000000000001000 x8 : 0000000000000000
[  122.636406] x7 : 0000000000000009 x6 : ffff2027c6e10000
[  122.641693] x5 : 0000000000000003 x4 : 0000000000000f44
[  122.646980] x3 : 00000000000ef130 x2 : 0000000000000002
[  122.652266] x1 : 0000000000000001 x0 : 0000000000000003
[  122.657554] Call trace:
[  122.659989]  __arm_lpae_map+0x1d0/0x2bc
[  122.663807]  __arm_lpae_map+0xf0/0x2bc
[  122.667537]  __arm_lpae_map+0xf0/0x2bc
[  122.671270]  __arm_lpae_map+0xf0/0x2bc
[  122.675003]  arm_lpae_map+0xdc/0x164
[  122.678563]  arm_smmu_map+0x18/0x28
[  122.682035]  __iommu_map+0xdc/0x17c
[  122.685508]  iommu_map_atomic+0x10/0x18
[  122.689325]  __iommu_dma_map+0xcc/0xe4
[  122.693058]  iommu_dma_map_page+0x80/0xc4
[  122.697050]  nvme_queue_rq+0x7dc/0x7fc
[  122.700781]  __blk_mq_try_issue_directly+0x108/0x1c0
[  122.705722]  blk_mq_request_issue_directly+0x40/0x64
[  122.710663]  blk_mq_try_issue_list_directly+0x5c/0xf0
[  122.715692]  blk_mq_sched_insert_requests+0x170/0x1d0
[  122.720721]  blk_mq_flush_plug_list+0x10c/0x158
[  122.725231]  blk_flush_plug_list+0xc4/0xd4
[  122.729308]  blk_finish_plug+0x30/0x40
[  122.733040]  blkdev_direct_IO+0x3d4/0x444
[  122.737034]  generic_file_read_iter+0x90/0xaf8
[  122.741458]  blkdev_read_iter+0x3c/0x54
[  122.745276]  aio_read+0xdc/0x138
[  122.748490]  io_submit_one+0x4ac/0xbf0
[  122.752221]  __arm64_sys_io_submit+0x16c/0x1f8
[  122.756645]  el0_svc_common.constprop.3+0xb8/0x170
[  122.761415]  do_el0_svc+0x70/0x88
[  122.764716]  el0_sync_handler+0xf0/0x12c
[  122.768620]  el0_sync+0x140/0x180
[  122.771918] ---[ end trace 2e5c7ee849d0ea30 ]---
[  122.776604] ------------[ cut here ]------------

and many more spewed out. And this is after the RCU stall report from a 
CPU locking up handling the NVMe hard interrupt (which has been seen on 
previous kernels).

> The explanation from
> Alexey in the reply to your mail makes complete sense to me.
> .

I meant that the original patch commit log had disassembly, etc to prove 
the worth, but now not much to show that the patch+fix is still better.

Thanks,
John

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

  reply	other threads:[~2020-05-06 16:27 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-28 18:45 [PATCH] nvme-pci: slimmer CQ head update Alexey Dobriyan
2020-02-29  5:53 ` Keith Busch
2020-05-06 11:03   ` John Garry
2020-05-06 12:47     ` Keith Busch
2020-05-06 13:24       ` Alexey Dobriyan
2020-05-06 13:44         ` John Garry
2020-05-06 14:01           ` Alexey Dobriyan
2020-05-06 14:35           ` Christoph Hellwig
2020-05-06 16:26             ` John Garry [this message]
2020-05-06 16:31               ` Will Deacon
2020-05-06 16:52                 ` Robin Murphy
2020-05-06 17:02                   ` John Garry
2020-05-07  8:18                     ` John Garry
2020-05-07 11:04                       ` Robin Murphy
2020-05-07 13:55                         ` John Garry
2020-05-07 14:23                           ` Keith Busch
2020-05-07 15:11                             ` John Garry
2020-05-07 15:35                               ` Keith Busch
2020-05-07 15:41                                 ` John Garry
2020-05-08 16:16                                   ` Keith Busch
2020-05-08 17:04                                     ` John Garry
2020-05-07 16:26                                 ` Robin Murphy
2020-05-07 17:35                                   ` Keith Busch
2020-05-07 17:44                                     ` Will Deacon
2020-05-07 18:06                                       ` Keith Busch
2020-05-08 11:40                                         ` Will Deacon
2020-05-08 14:07                                           ` Keith Busch
2020-05-08 15:34                                             ` Keith Busch
2020-05-06 14:44         ` Keith Busch
2020-05-07 15:58           ` Keith Busch
2020-05-07 20:07             ` [PATCH] nvme-pci: fix "slimmer CQ head update" Alexey Dobriyan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4155a814-798c-0c7e-5433-daf719c0345c@huawei.com \
    --to=john.garry@huawei.com \
    --cc=adobriyan@gmail.com \
    --cc=axboe@fb.com \
    --cc=hch@lst.de \
    --cc=kbusch@kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=robin.murphy@arm.com \
    --cc=sagi@grimberg.me \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.