From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933799AbcBBRUM (ORCPT ); Tue, 2 Feb 2016 12:20:12 -0500 Received: from mail-wm0-f49.google.com ([74.125.82.49]:35368 "EHLO mail-wm0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933749AbcBBRUG (ORCPT ); Tue, 2 Feb 2016 12:20:06 -0500 Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended To: Keith Busch References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com> <56AF8DB5.70206@fb.com> <56B0A401.30306@dev.mellanox.co.il> <20160202142756.GA10690@localhost.localdomain> <56B0BE26.6070208@dev.mellanox.co.il> <20160202144648.GA10728@localhost.localdomain> Cc: Wenbo Wang , Jens Axboe , Wenbo Wang , "Wenwei.Tao" , "linux-kernel@vger.kernel.org" , "linux-nvme@lists.infradead.org" From: Sagi Grimberg Message-ID: <56B0E541.1000907@dev.mellanox.co.il> Date: Tue, 2 Feb 2016 19:20:01 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <20160202144648.GA10728@localhost.localdomain> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > We free the transfer buffers when a command is cancelled. The controller, > however, may still own the command and may try to write to them. We > have to fence the controller off from being able to do that, so we can't > cancel inflight commands while the PCI device is still bus master enabled. > > In a perfect world, we could trust in disabling with NVMe registers, > but sometimes we can't rely on that. OK, I wasn't aware that we cannot rely on that. So it looks like we cannot change the ordering. So this leaves us with the need to guarantee that no queue_rq is inflight before we unmap. From mboxrd@z Thu Jan 1 00:00:00 1970 From: sagig@dev.mellanox.co.il (Sagi Grimberg) Date: Tue, 2 Feb 2016 19:20:01 +0200 Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended In-Reply-To: <20160202144648.GA10728@localhost.localdomain> References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com> <56AF8DB5.70206@fb.com> <56B0A401.30306@dev.mellanox.co.il> <20160202142756.GA10690@localhost.localdomain> <56B0BE26.6070208@dev.mellanox.co.il> <20160202144648.GA10728@localhost.localdomain> Message-ID: <56B0E541.1000907@dev.mellanox.co.il> > We free the transfer buffers when a command is cancelled. The controller, > however, may still own the command and may try to write to them. We > have to fence the controller off from being able to do that, so we can't > cancel inflight commands while the PCI device is still bus master enabled. > > In a perfect world, we could trust in disabling with NVMe registers, > but sometimes we can't rely on that. OK, I wasn't aware that we cannot rely on that. So it looks like we cannot change the ordering. So this leaves us with the need to guarantee that no queue_rq is inflight before we unmap.