Linux-SCSI Archive on lore.kernel.org
 help / color / Atom feed
From: Hannes Reinecke <hare@suse.de>
To: Keith Busch <kbusch@kernel.org>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>,
	Tim Walker <tim.t.walker@seagate.com>,
	Damien Le Moal <Damien.LeMoal@wdc.com>,
	Ming Lei <ming.lei@redhat.com>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	linux-scsi <linux-scsi@vger.kernel.org>,
	"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>
Subject: Re: [LSF/MM/BPF TOPIC] NVMe HDD
Date: Fri, 14 Feb 2020 17:04:25 +0100
Message-ID: <d043a58d-6584-1792-4433-ac2cc39526ca@suse.de> (raw)
In-Reply-To: <20200214144007.GD9819@redsun51.ssa.fujisawa.hgst.com>

On 2/14/20 3:40 PM, Keith Busch wrote:
> On Fri, Feb 14, 2020 at 08:32:57AM +0100, Hannes Reinecke wrote:
>> On 2/13/20 5:17 AM, Martin K. Petersen wrote:
>>> People often artificially lower the queue depth to avoid timeouts. The
>>> default timeout is 30 seconds from an I/O is queued. However, many
>>> enterprise applications set the timeout to 3-5 seconds. Which means that
>>> with deep queues you'll quickly start seeing timeouts if a drive
>>> temporarily is having issues keeping up (media errors, excessive spare
>>> track seeks, etc.).
>>>
>>> Well-behaved devices will return QF/TSF if they have transient resource
>>> starvation or exceed internal QoS limits. QF will cause the SCSI stack
>>> to reduce the number of I/Os in flight. This allows the drive to recover
>>> from its congested state and reduces the potential of application and
>>> filesystem timeouts.
>>>
>> This may even be a chance to revisit QoS / queue busy handling.
>> NVMe has this SQ head pointer mechanism which was supposed to handle
>> this kind of situations, but to my knowledge no-one has been
>> implementing it.
>> Might be worthwhile revisiting it; guess NVMe HDDs would profit from that.
> 
> We don't need that because we don't allocate enough tags to potentially
> wrap the tail past the head. If you can allocate a tag, the queue is not
> full. And convesely, no tag == queue full.
> 
It's not a problem on our side.
It's a problem on the target/controller side.
The target/controller might have a need to throttle I/O (due to QoS 
settings or competing resources from other hosts), but currently no 
means of signalling that to the host.
Which, incidentally, is the underlying reason for the DNR handling 
discussion we had; NetApp tried to model QoS by sending "Namespace not 
ready" without the DNR bit set, which of course is a totally different 
use-case as the typical 'Namespace not ready' response we get (with the 
DNR bit set) when a namespace was unmapped.

And that is where SQ head pointer updates comes in; it would allow the 
controller to signal back to the host that it should hold off sending 
I/O for a bit.
So this could / might be used for NVMe HDDs, too, which also might have 
a need to signal back to the host that I/Os should be throttled...

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

  reply index

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-10 19:20 Tim Walker
2020-02-10 20:43 ` Keith Busch
2020-02-10 22:25   ` Finn Thain
2020-02-11 12:28 ` Ming Lei
2020-02-11 19:01   ` Tim Walker
2020-02-12  1:47     ` Damien Le Moal
2020-02-12 22:03       ` Ming Lei
2020-02-13  2:40         ` Damien Le Moal
2020-02-13  7:53           ` Ming Lei
2020-02-13  8:24             ` Damien Le Moal
2020-02-13  8:34               ` Ming Lei
2020-02-13 16:30                 ` Keith Busch
2020-02-14  0:40                   ` Ming Lei
2020-02-13  3:02       ` Martin K. Petersen
2020-02-13  3:12         ` Tim Walker
2020-02-13  4:17           ` Martin K. Petersen
2020-02-14  7:32             ` Hannes Reinecke
2020-02-14 14:40               ` Keith Busch
2020-02-14 16:04                 ` Hannes Reinecke [this message]
2020-02-14 17:05                   ` Keith Busch
2020-02-14  0:35         ` Ming Lei
2020-02-12 21:52     ` Ming Lei

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d043a58d-6584-1792-4433-ac2cc39526ca@suse.de \
    --to=hare@suse.de \
    --cc=Damien.LeMoal@wdc.com \
    --cc=kbusch@kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=ming.lei@redhat.com \
    --cc=tim.t.walker@seagate.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-SCSI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-scsi/0 linux-scsi/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-scsi linux-scsi/ https://lore.kernel.org/linux-scsi \
		linux-scsi@vger.kernel.org
	public-inbox-index linux-scsi

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-scsi


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git