Re: [PATCH net-next 1/1] ice: Improve MSI-X vector enablement fallback logic

From: Jakub Kicinski <kuba@kernel.org>
To: Tony Nguyen <anthony.l.nguyen@intel.com>
Cc: davem@davemloft.net, Brett Creeley <brett.creeley@intel.com>,
	netdev@vger.kernel.org, sassmann@redhat.com,
	Tony Brelinski <tonyx.brelinski@intel.com>
Subject: Re: [PATCH net-next 1/1] ice: Improve MSI-X vector enablement fallback logic
Date: Thu, 14 Jan 2021 16:42:52 -0800	[thread overview]
Message-ID: <20210114164252.74c1cf18@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <20210113234226.3638426-1-anthony.l.nguyen@intel.com>

On Wed, 13 Jan 2021 15:42:26 -0800 Tony Nguyen wrote:
> From: Brett Creeley <brett.creeley@intel.com>
> 
> The current MSI-X enablement logic either tries to enable best-case
> MSI-X vectors and if that fails we only support a bare-minimum set.
> This is not very flexible and the current logic is broken when actually
> allocating and reserving MSI-X in the driver. Fix this by improving the
> fall-back logic and fixing the allocation/reservation of MSI-X when the
> best-case MSI-X vectors are not received from the OS.
> 
> The new fall-back logic is described below with each [#] being an
> attempt at enabling a certain number of MSI-X. If any of the steps
> succeed, then return the number of MSI-X enabled from
> ice_ena_msix_range(). If any of the attempts fail, then goto the next
> step.
> 
> Attempt [0]: Enable the best-case scenario MSI-X vectors.
> 
> Attempt [1]: Enable MSI-X vectors with the number of pf->num_lan_msix
> reduced by a factor of 2 from the previous attempt (i.e.
> num_online_cpus() / 2).
> 
> Attempt [2]: Same as attempt [1], except reduce by a factor of 4.
> 
> Attempt [3]: Enable the bare-minimum MSI-X vectors.
> 
> Also, if the adjusted_base_msix ever hits the minimum required for LAN,
> then just set the needed MSI-X for that feature to the minimum
> (similar to attempt [3]).

I don't really get why you switch to this manual "exponential back-off"
rather than continuing to use pci_enable_msix_range(), but fixing the
capping to ICE_MIN_LAN_VECS.

> To fix the allocation/reservation of MSI-X, the PF VSI needs to take
> into account the pf->num_lan_msix available and only allocate up to that
> many MSI-X vectors. To do this, limit the number of Tx and Rx queues
> based on pf->num_lan_msix. This is done because we don't want more than
> 1 Tx/Rx queue per interrupt due to performance concerns. Also, limit the
> number of MSI-X based on pf->num_lan_msix available.
> 
> Also, prevent users from enabling more combined queues than there are
> MSI-X available via ethtool.

Right, this part sounds like a fix, and should be a separate patch
against net, not net-next.

> Fixes: 152b978a1f90 ("ice: Rework ice_ena_msix_range")
> Signed-off-by: Brett Creeley <brett.creeley@intel.com>
> Tested-by: Tony Brelinski <tonyx.brelinski@intel.com>
> Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>