From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S933397AbcLGWeb (ORCPT <rfc822;w@1wt.eu>);
        Wed, 7 Dec 2016 17:34:31 -0500
Received: from mga03.intel.com ([134.134.136.65]:54872 "EHLO mga03.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S932892AbcLGWea (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 7 Dec 2016 17:34:30 -0500
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.33,315,1477983600"; 
   d="scan'208";a="40126251"
Date: Wed, 7 Dec 2016 17:44:14 -0500
From: Keith Busch <keith.busch@intel.com>
To: Dan Streetman <ddstreet@ieee.org>
Cc: Jens Axboe <axboe@fb.com>, linux-nvme@lists.infradead.org,
        linux-kernel@vger.kernel.org,
        Dan Streetman <dan.streetman@canonical.com>
Subject: Re: [PATCH] nvme: use the correct msix vector for each queue
Message-ID: <20161207224414.GE22478@localhost.localdomain>
References: <20161207220348.8572-1-ddstreet@ieee.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161207220348.8572-1-ddstreet@ieee.org>
User-Agent: Mutt/1.7.0 (2016-08-17)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Dec 07, 2016 at 05:03:48PM -0500, Dan Streetman wrote:
> Change each queue's cq_vector to match its qid, instead of qid - 1.
> 
> The first queue is always the admin queue, and the remaining queues are
> I/O queues.  The interrupt vectors they use are all in the same array,
> however, the vector indexes for the admin and I/O queues are setup
> differently; the admin queue's cq_vector is manually set to 0, while
> each I/O queue's cq_vector is set to qid - 1.  Since the admin queue
> is qid 0, and the I/O queues start at qid 1, using qid - 1 is wrong for the
> I/O queues, as it makes the first I/O queue (qid 1) share the vector from
> the admin queue (qid 0), and no queue uses the last interrupt vector.
> Instead, each I/O queue should set their cq_vector to qid.

pci_alloc_irq_vectors doesn't know you intend to make the first
vector special, so it's going to come up with a CPU affinity from
blk_mq_pci_map_queues that clashes with what you've programmed in the
IO completion queues.