All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@fb.com>
To: Ming Lei <tom.leiming@gmail.com>
Cc: "Matias Bjørling" <m@bjorling.me>,
	"Matthew Wilcox" <willy@linux.intel.com>,
	"Keith Busch" <keith.busch@intel.com>,
	"Sam Bradshaw (sbradshaw)" <sbradshaw@micron.com>,
	"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
	linux-nvme <linux-nvme@lists.infradead.org>
Subject: Re: [PATCH v6] NVMe: conversion to blk-mq
Date: Mon, 9 Jun 2014 09:10:28 -0600	[thread overview]
Message-ID: <5395CE64.6040402@fb.com> (raw)
In-Reply-To: <CACVXFVPGO9vsOv_uVa0+5AdBduu1RJ0Fmg8XPx1QgJAvnjKj_g@mail.gmail.com>

On 2014-06-09 09:07, Ming Lei wrote:
> On Mon, Jun 9, 2014 at 10:08 PM, Jens Axboe <axboe@fb.com> wrote:
>> On 2014-06-08 22:35, Ming Lei wrote:
>>>
>>> On Fri, Jun 6, 2014 at 8:20 PM, Matias Bjørling <m@bjorling.me> wrote:
>>>>
>>>> This converts the current NVMe driver to utilize the blk-mq layer.
>>>
>>>
>>> Looks it can't be applied cleanly against 3.15-rc8 + Jens's for-linux
>>> branch, when I fix the conflict manually, below failure is triggered:
>>>
>>> [  487.696057] nvme 0000:00:07.0: Cancelling I/O 202 QID 1
>>> [  487.699005] nvme 0000:00:07.0: Aborting I/O 202 QID 1
>>> [  487.704074] nvme 0000:00:07.0: Cancelling I/O 202 QID 1
>>> [  487.717881] nvme 0000:00:07.0: Aborting I/O 202 QID 1
>>> [  487.736093] end_request: I/O error, dev nvme0n1, sector 91532352
>>> [  487.747378] nvme 0000:00:07.0: completed id 0 twice on queue 0
>>>
>>>
>>> when running fio randread(libaio, iodepth:64) with more than 3 jobs.
>>>
>>> And looks no such failure when jobs is 1 or 2.
>>
>>
>> It's a known issue, make sure you have this patch:
>>
>> https://urldefense.proofpoint.com/v1/url?u=http://git.kernel.dk/?p%3Dlinux-block.git%3Ba%3Dcommit%3Bh%3Df6be4fb4bcb396fc3b1c134b7863351972de081f&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=3JMVyziIyZtZ5cv9eWNLwQ%3D%3D%0A&m=04gqJgKKgVgGynQnyGFy4OmxWZ5%2FJ%2FwBiXQH3K%2BEZ4U%3D%0A&s=b137111b72079e93492763bfa23fd3458cd9b287a5eb3b68f61b69c9cda6b7a5
>
> Looks assignment to 'rq->deadline' in blk_mq_start_request()
> may be removed,  since blk_add_timer() already did that.
>
> But that isn't the only issue.
>
> Another one is that req->start_time isn't set without IO_STAT in
> nvme, which may cause similar issue too, so req->start_time
> can be set in nvme's queue_rq callback.
>
> My real issue should be nvme specific: NVME_INTERNAL_DEV_ERROR(0x6) is
> returned from device
> with the conversion patch, but no such issue in current bio mode.

You are right, I didn't get to the bottom of the thread first.
Might be better to just have ->start_time set always, regardless of io 
stats being enabled or not.

-- 
Jens Axboe


WARNING: multiple messages have this Message-ID (diff)
From: axboe@fb.com (Jens Axboe)
Subject: [PATCH v6] NVMe: conversion to blk-mq
Date: Mon, 9 Jun 2014 09:10:28 -0600	[thread overview]
Message-ID: <5395CE64.6040402@fb.com> (raw)
In-Reply-To: <CACVXFVPGO9vsOv_uVa0+5AdBduu1RJ0Fmg8XPx1QgJAvnjKj_g@mail.gmail.com>

On 2014-06-09 09:07, Ming Lei wrote:
> On Mon, Jun 9, 2014@10:08 PM, Jens Axboe <axboe@fb.com> wrote:
>> On 2014-06-08 22:35, Ming Lei wrote:
>>>
>>> On Fri, Jun 6, 2014@8:20 PM, Matias Bj?rling <m@bjorling.me> wrote:
>>>>
>>>> This converts the current NVMe driver to utilize the blk-mq layer.
>>>
>>>
>>> Looks it can't be applied cleanly against 3.15-rc8 + Jens's for-linux
>>> branch, when I fix the conflict manually, below failure is triggered:
>>>
>>> [  487.696057] nvme 0000:00:07.0: Cancelling I/O 202 QID 1
>>> [  487.699005] nvme 0000:00:07.0: Aborting I/O 202 QID 1
>>> [  487.704074] nvme 0000:00:07.0: Cancelling I/O 202 QID 1
>>> [  487.717881] nvme 0000:00:07.0: Aborting I/O 202 QID 1
>>> [  487.736093] end_request: I/O error, dev nvme0n1, sector 91532352
>>> [  487.747378] nvme 0000:00:07.0: completed id 0 twice on queue 0
>>>
>>>
>>> when running fio randread(libaio, iodepth:64) with more than 3 jobs.
>>>
>>> And looks no such failure when jobs is 1 or 2.
>>
>>
>> It's a known issue, make sure you have this patch:
>>
>> https://urldefense.proofpoint.com/v1/url?u=http://git.kernel.dk/?p%3Dlinux-block.git%3Ba%3Dcommit%3Bh%3Df6be4fb4bcb396fc3b1c134b7863351972de081f&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=3JMVyziIyZtZ5cv9eWNLwQ%3D%3D%0A&m=04gqJgKKgVgGynQnyGFy4OmxWZ5%2FJ%2FwBiXQH3K%2BEZ4U%3D%0A&s=b137111b72079e93492763bfa23fd3458cd9b287a5eb3b68f61b69c9cda6b7a5
>
> Looks assignment to 'rq->deadline' in blk_mq_start_request()
> may be removed,  since blk_add_timer() already did that.
>
> But that isn't the only issue.
>
> Another one is that req->start_time isn't set without IO_STAT in
> nvme, which may cause similar issue too, so req->start_time
> can be set in nvme's queue_rq callback.
>
> My real issue should be nvme specific: NVME_INTERNAL_DEV_ERROR(0x6) is
> returned from device
> with the conversion patch, but no such issue in current bio mode.

You are right, I didn't get to the bottom of the thread first.
Might be better to just have ->start_time set always, regardless of io 
stats being enabled or not.

-- 
Jens Axboe

  reply	other threads:[~2014-06-09 15:11 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-06-06 12:20 [PATCH v6] conversion to blk-mq Matias Bjørling
2014-06-06 12:20 ` Matias Bjørling
2014-06-06 12:20 ` [PATCH v6] NVMe: " Matias Bjørling
2014-06-06 12:20   ` Matias Bjørling
2014-06-09  4:35   ` Ming Lei
2014-06-09  4:35     ` Ming Lei
2014-06-09  6:00     ` Matias Bjørling
2014-06-09  6:00       ` Matias Bjørling
2014-06-09  7:50       ` Ming Lei
2014-06-09  7:50         ` Ming Lei
2014-06-09  7:53         ` Ming Lei
2014-06-09  7:53           ` Ming Lei
2014-06-09 10:40           ` Ming Lei
2014-06-09 10:40             ` Ming Lei
2014-06-12  0:58             ` Ming Lei
2014-06-12  0:58               ` Ming Lei
2014-06-09 14:08     ` Jens Axboe
2014-06-09 14:08       ` Jens Axboe
2014-06-09 14:59       ` Christoph Hellwig
2014-06-09 14:59         ` Christoph Hellwig
2014-06-09 15:27         ` Jens Axboe
2014-06-09 15:27           ` Jens Axboe
2014-06-09 15:35           ` Ming Lei
2014-06-09 15:35             ` Ming Lei
2014-06-10 11:35             ` Christoph Hellwig
2014-06-10 11:35               ` Christoph Hellwig
2014-06-10 11:55               ` Ming Lei
2014-06-10 11:55                 ` Ming Lei
2014-06-10 12:40                 ` Christoph Hellwig
2014-06-10 12:40                   ` Christoph Hellwig
2014-06-09 15:07       ` Ming Lei
2014-06-09 15:07         ` Ming Lei
2014-06-09 15:10         ` Jens Axboe [this message]
2014-06-09 15:10           ` Jens Axboe
2014-06-09 15:15           ` Ming Lei
2014-06-09 15:15             ` Ming Lei
2014-06-09 15:40             ` Jens Axboe
2014-06-09 15:40               ` Jens Axboe
2014-06-10 19:42               ` Matias Bjørling
2014-06-10 19:42                 ` Matias Bjørling

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5395CE64.6040402@fb.com \
    --to=axboe@fb.com \
    --cc=keith.busch@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=m@bjorling.me \
    --cc=sbradshaw@micron.com \
    --cc=tom.leiming@gmail.com \
    --cc=willy@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.