From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jiri Pirko Subject: Re: [net-next 1/6] net/dcb: Add dcbnl buffer attribute Date: Wed, 23 May 2018 11:43:31 +0200 Message-ID: <20180523094331.GC3046@nanopsycho> References: <20180521210502.11082-1-saeedm@mellanox.com> <20180521210502.11082-2-saeedm@mellanox.com> <20180521222026.4f54f479@cakuba> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Saeed Mahameed , "David S. Miller" , netdev@vger.kernel.org, Huy Nguyen , Or Gerlitz To: Jakub Kicinski Return-path: Received: from mail-wm0-f44.google.com ([74.125.82.44]:53203 "EHLO mail-wm0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754411AbeEWJno (ORCPT ); Wed, 23 May 2018 05:43:44 -0400 Received: by mail-wm0-f44.google.com with SMTP id w194-v6so7147866wmf.2 for ; Wed, 23 May 2018 02:43:43 -0700 (PDT) Content-Disposition: inline In-Reply-To: <20180521222026.4f54f479@cakuba> Sender: netdev-owner@vger.kernel.org List-ID: Tue, May 22, 2018 at 07:20:26AM CEST, jakub.kicinski@netronome.com wrote: >On Mon, 21 May 2018 14:04:57 -0700, Saeed Mahameed wrote: >> From: Huy Nguyen >> >> In this patch, we add dcbnl buffer attribute to allow user >> change the NIC's buffer configuration such as priority >> to buffer mapping and buffer size of individual buffer. >> >> This attribute combined with pfc attribute allows advance user to >> fine tune the qos setting for specific priority queue. For example, >> user can give dedicated buffer for one or more prirorities or user >> can give large buffer to certain priorities. >> >> We present an use case scenario where dcbnl buffer attribute configured >> by advance user helps reduce the latency of messages of different sizes. >> >> Scenarios description: >> On ConnectX-5, we run latency sensitive traffic with >> small/medium message sizes ranging from 64B to 256KB and bandwidth sensitive >> traffic with large messages sizes 512KB and 1MB. We group small, medium, >> and large message sizes to their own pfc enables priorities as follow. >> Priorities 1 & 2 (64B, 256B and 1KB) >> Priorities 3 & 4 (4KB, 8KB, 16KB, 64KB, 128KB and 256KB) >> Priorities 5 & 6 (512KB and 1MB) >> >> By default, ConnectX-5 maps all pfc enabled priorities to a single >> lossless fixed buffer size of 50% of total available buffer space. The >> other 50% is assigned to lossy buffer. Using dcbnl buffer attribute, >> we create three equal size lossless buffers. Each buffer has 25% of total >> available buffer space. Thus, the lossy buffer size reduces to 25%. Priority >> to lossless buffer mappings are set as follow. >> Priorities 1 & 2 on lossless buffer #1 >> Priorities 3 & 4 on lossless buffer #2 >> Priorities 5 & 6 on lossless buffer #3 >> >> We observe improvements in latency for small and medium message sizes >> as follows. Please note that the large message sizes bandwidth performance is >> reduced but the total bandwidth remains the same. >> 256B message size (42 % latency reduction) >> 4K message size (21% latency reduction) >> 64K message size (16% latency reduction) >> >> Signed-off-by: Huy Nguyen >> Signed-off-by: Saeed Mahameed > >On a cursory look this bares a lot of resemblance to devlink shared >buffer configuration ABI. Did you look into using that? > >Just to be clear devlink shared buffer ABIs don't require representors >and "switchdev mode". If the CX5 buffer they are trying to utilize here is per port and not a shared one, it would seem ok for me to not have it in "devlink sb".