From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEFE1C7618F for ; Thu, 18 Jul 2019 14:42:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7E6BD21849 for ; Thu, 18 Jul 2019 14:42:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727730AbfGROm5 (ORCPT ); Thu, 18 Jul 2019 10:42:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37236 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726040AbfGROm5 (ORCPT ); Thu, 18 Jul 2019 10:42:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 35C6DC065134; Thu, 18 Jul 2019 14:42:56 +0000 (UTC) Received: from redhat.com (ovpn-120-147.rdu2.redhat.com [10.10.120.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 972C45D739; Thu, 18 Jul 2019 14:42:48 +0000 (UTC) Date: Thu, 18 Jul 2019 10:42:47 -0400 From: "Michael S. Tsirkin" To: Jason Wang Cc: ? jiang , "davem@davemloft.net" , "ast@kernel.org" , "daniel@iogearbox.net" , "jakub.kicinski@netronome.com" , "hawk@kernel.org" , "john.fastabend@gmail.com" , "kafai@fb.com" , "songliubraving@fb.com" , "yhs@fb.com" , "virtualization@lists.linux-foundation.org" , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "xdp-newbies@vger.kernel.org" , "bpf@vger.kernel.org" , "jiangran.jr@alibaba-inc.com" Subject: Re: [PATCH] virtio-net: parameterize min ring num_free for virtio receive Message-ID: <20190718103641-mutt-send-email-mst@kernel.org> References: <20190718085836-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 18 Jul 2019 14:42:56 +0000 (UTC) Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Thu, Jul 18, 2019 at 10:01:05PM +0800, Jason Wang wrote: > > On 2019/7/18 下午9:04, Michael S. Tsirkin wrote: > > On Thu, Jul 18, 2019 at 12:55:50PM +0000, ? jiang wrote: > > > This change makes ring buffer reclaim threshold num_free configurable > > > for better performance, while it's hard coded as 1/2 * queue now. > > > According to our test with qemu + dpdk, packet dropping happens when > > > the guest is not able to provide free buffer in avail ring timely. > > > Smaller value of num_free does decrease the number of packet dropping > > > during our test as it makes virtio_net reclaim buffer earlier. > > > > > > At least, we should leave the value changeable to user while the > > > default value as 1/2 * queue is kept. > > > > > > Signed-off-by: jiangkidd > > That would be one reason, but I suspect it's not the > > true one. If you need more buffer due to jitter > > then just increase the queue size. Would be cleaner. > > > > > > However are you sure this is the reason for > > packet drops? Do you see them dropped by dpdk > > due to lack of space in the ring? As opposed to > > by guest? > > > > > > Besides those, this patch depends on the user to choose a suitable threshold > which is not good. You need either a good value with demonstrated numbers or > something smarter. > > Thanks I do however think that we have a problem right now: try_fill_recv can take up a long time during which net stack does not run at all. Imagine a 1K queue - we are talking 512 packets. That's exceessive. napi poll weight solves a similar problem, so it might make sense to cap this at napi_poll_weight. Which will allow tweaking it through a module parameter as a side effect :) Maybe just do NAPI_POLL_WEIGHT. Need to be careful though: queues can also be small and I don't think we want to exceed queue size / 2, or maybe queue size - napi_poll_weight. Definitely must not exceed the full queue size. -- MST