From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753252AbcEPDAO (ORCPT ); Sun, 15 May 2016 23:00:14 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:40425 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751921AbcEPDAM (ORCPT ); Sun, 15 May 2016 23:00:12 -0400 Subject: Re: [PATCH] ixgbe: take online CPU number as MQ max limit when alloc_etherdev_mq() To: Alexander Duyck References: <1463118995-31763-1-git-send-email-ethan.zhao@oracle.com> Cc: Jeff Kirsher , "Brandeburg, Jesse" , shannon nelson , Carolyn Wyborny , "Skidmore, Donald C" , Bruce W Allan , John Ronciak , Mitch Williams , intel-wired-lan , Netdev , "linux-kernel@vger.kernel.org" , Ethan Zhao From: ethan zhao Organization: Oracle Corporation Message-ID: <573937A7.9010405@oracle.com> Date: Mon, 16 May 2016 10:59:51 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: aserv0021.oracle.com [141.146.126.233] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Alexander, On 2016/5/14 0:46, Alexander Duyck wrote: > On Thu, May 12, 2016 at 10:56 PM, Ethan Zhao wrote: >> Allocating 64 Tx/Rx as default doesn't benefit perfomrnace when less >> CPUs were assigned. especially when DCB is enabled, so we should take >> num_online_cpus() as top limit, and aslo to make sure every TC has >> at least one queue, take the MAX_TRAFFIC_CLASS as bottom limit of queues >> number. >> >> Signed-off-by: Ethan Zhao > What is the harm in allowing the user to specify up to 64 queues if > they want to? Also what is your opinion based on? In the case of RSS There is no module parameter to specify queue number in this upstream ixgbe driver. for what to specify more queues than num_online_cpus() via ethtool ? I couldn't figure out the benefit to do that. But if DCB is turned on after loading, the queues would be 64/64, that doesn't make sense if only 16 CPUs assigned. > traffic the upper limit is only 16 on older NICs, but last I knew the > latest X550 can support more queues for RSS. Have you only been > testing on older NICs or did you test on the latest hardware as well? More queues for RSS than num_online_cpus() could bring better performance ? Test result shows false result. even memory cost is not an issue for most of the expensive servers, but not for all. > > If you want to control the number of queues allocated in a given > configuration you should look at the code over in the ixgbe_lib.c, not Yes, RSS, RSS with SRIOV, FCoE, DCB etc uses different queues calculation algorithm. But they all take the dev queues allocated in alloc_etherdev_mq() as upper limit. If we set 64 as default here, DCB would says "oh, there is 64 there, I could use it" > ixgbe_main.c. All you are doing with this patch is denying the user > choice with this change as they then are not allowed to set more Yes, it is purposed to deny configuration that doesn't benefit. > queues. Even if they find your decision was wrong for their > configuration. > > - Alex > Thanks, Ethan