From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Fastabend via iovisor-dev Subject: Re: [PATCH RFC 11/11] net/mlx5e: XDP TX xmit more Date: Wed, 7 Sep 2016 19:58:50 -0700 Message-ID: <57D0D3EA.1090004@gmail.com> References: <1473252152-11379-1-git-send-email-saeedm@mellanox.com> <1473252152-11379-12-git-send-email-saeedm@mellanox.com> <1473259302.10725.31.camel@edumazet-glaptop3.roam.corp.google.com> <1473262379.10725.42.camel@edumazet-glaptop3.roam.corp.google.com> <20160907202234.55e18ef3@redhat.com> Reply-To: John Fastabend Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Eric Dumazet , Linux Netdev List , iovisor-dev , Jamal Hadi Salim , Saeed Mahameed , Eric Dumazet , Tom Herbert To: Jesper Dangaard Brouer , Saeed Mahameed Return-path: In-Reply-To: <20160907202234.55e18ef3-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iovisor-dev-bounces-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org Errors-To: iovisor-dev-bounces-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org List-Id: netdev.vger.kernel.org On 16-09-07 11:22 AM, Jesper Dangaard Brouer wrote: > > On Wed, 7 Sep 2016 19:57:19 +0300 Saeed Mahameed wrote: >> On Wed, Sep 7, 2016 at 6:32 PM, Eric Dumazet wrote: >>> On Wed, 2016-09-07 at 18:08 +0300, Saeed Mahameed wrote: >>>> On Wed, Sep 7, 2016 at 5:41 PM, Eric Dumazet wrote: >>>>> On Wed, 2016-09-07 at 15:42 +0300, Saeed Mahameed wrote: > [...] >>> >>> Only if a qdisc is present and pressure is high enough. >>> >>> But in a forwarding setup, we likely receive at a lower rate than the >>> NIC can transmit. > > Yes, I can confirm this happens in my experiments. > >>> >> >> Jesper has a similar Idea to make the qdisc think it is under >> pressure, when the device TX ring is idle most of the time, i think >> his idea can come in handy here. I am not fully involved in the >> details, maybe he can elaborate more. >> >> But if it works, it will be transparent to napi, and xmit more will >> happen by design. > > Yes. I have some ideas around getting more bulking going from the qdisc > layer, by having the drivers provide some feedback to the qdisc layer > indicating xmit_more should be possible. This will be a topic at the > Network Performance Workshop[1] at NetDev 1.2, I have will hopefully > challenge people to come up with a good solution ;-) > One thing I've noticed but haven't yet actually analyzed much is if I shrink the nic descriptor ring size to only be slightly larger than the qdisc layer bulking size I get more bulking and better perf numbers. At least on microbenchmarks. The reason being the nic pushes back more on the qdisc. So maybe a case for making the ring size in the NIC some factor of the expected number of queues feeding the descriptor ring. .John