All of lore.kernel.org
 help / color / mirror / Atom feed
From: Junjie Chen <junjie.j.chen@intel.com>
To: maxime.coquelin@redhat.com, jianfeng.tan@intel.com,
	john.mcnamara@intel.com, marko.kovacevic@intel.com
Cc: dev@dpdk.org, Junjie Chen <junjie.j.chen@intel.com>
Subject: [PATCH] doc/vhost: update zero copy perfromance tip
Date: Thu, 26 Apr 2018 16:13:20 +0800	[thread overview]
Message-ID: <1524730400-252654-1-git-send-email-junjie.j.chen@intel.com> (raw)

In VM2NIC case zero copy may need some tuning to get best performance. This
patch describes the zero copy starved case and provides a tuning tip.

Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index ef9a178..92dcdb5 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -65,14 +65,18 @@ The following is an overview of some key Vhost API functions:
     * zero copy is really good for VM2VM case. For iperf between two VMs, the
       boost could be above 70% (when TSO is enableld).
 
-    * for VM2NIC case, the ``nb_tx_desc`` has to be small enough: <= 64 if virtio
-      indirect feature is not enabled and <= 128 if it is enabled.
-
-      This is because when dequeue zero copy is enabled, guest Tx used vring will
-      be updated only when corresponding mbuf is freed. Thus, the nb_tx_desc
-      has to be small enough so that the PMD driver will run out of available
-      Tx descriptors and free mbufs timely. Otherwise, guest Tx vring would be
-      starved.
+    * For zero copy in VM2NIC case, guest Tx used vring may be starved if the
+      PMD driver consume the mbuf but not release them timely.
+
+      For example, i40e driver has an optimization to maximum NIC pipeline which
+      postpones returning transmitted mbuf until only tx_free_threshold free
+      descs left. The virtio TX used ring will be starved if the formula
+      (num_i40e_tx_desc - num_virtio_tx_desc > tx_free_threshold) is true, since
+      i40e will not return back mbuf.
+
+      A performance tip for tuning zero copy in VM2NIC case is to adjust the
+      frequency of mbuf free (i.e. adjust tx_free_threshold of i40e driver) to
+      balance consumer and producer.
 
     * Guest memory should be backended with huge pages to achieve better
       performance. Using 1G page size is the best.
-- 
1.8.3.1

             reply	other threads:[~2018-04-26  8:17 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-26  8:13 Junjie Chen [this message]
2018-05-04 12:49 ` doc/vhost: update zero copy perfromance tip Maxime Coquelin
2018-05-04 15:12 ` [PATCH] " Maxime Coquelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1524730400-252654-1-git-send-email-junjie.j.chen@intel.com \
    --to=junjie.j.chen@intel.com \
    --cc=dev@dpdk.org \
    --cc=jianfeng.tan@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=maxime.coquelin@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.