From: Zoltan Kiss <zoltan.kiss@citrix.com>
To: Wei Liu <wei.liu2@citrix.com>
Cc: <ian.campbell@citrix.com>, <xen-devel@lists.xenproject.org>,
<netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<jonathan.davies@citrix.com>
Subject: Re: [PATCH net-next v2 9/9] xen-netback: Aggregate TX unmap operations
Date: Mon, 16 Dec 2013 16:30:02 +0000 [thread overview]
Message-ID: <52AF2A8A.50404@citrix.com> (raw)
In-Reply-To: <20131213154435.GP21900@zion.uk.xensource.com>
On 13/12/13 15:44, Wei Liu wrote:
> On Thu, Dec 12, 2013 at 11:48:17PM +0000, Zoltan Kiss wrote:
>> Unmapping causes TLB flushing, therefore we should make it in the largest
>> possible batches. However we shouldn't starve the guest for too long. So if
>> the guest has space for at least two big packets and we don't have at least a
>> quarter ring to unmap, delay it for at most 1 milisec.
>>
>
> Is this solution temporary or permanent? If it is permanent would it
> make sense to make these parameter tunable?
Well, I'm not entirely sure yet this is the best way to do this, so in
this sense it's temporary. But generally we should do some sort of
batching, as TLB flush cannot be avoided every time. If we settle on
something we should make the tunable parameters tunable.
The problem is that it is a thin red line we should find here. My first
approach was that I left the tx_dealloc_work_todo as it was, and after
the thread woke up but before anything were done I made it sleep for 50
ns and measured how fast the guest is running out of free slots:
if (kthread_should_stop())
break;
+i=0;
+do {
+ ++i;
+ prev_free_slots = nr_free_slots(&vif->tx);
+ __set_current_state(TASK_UNINTERRUPTIBLE);
+ rc = schedule_hrtimeout_range(&tx_dealloc_delay_ktime, 10,
HRTIMER_MODE_REL);
+ if (rc) trace_printk("%s sleep were interrupted! %d\n",vif->dev->name,
rc);
+ curr_free_slots = nr_free_slots(&vif->tx);
+} while ( (curr_free_slots < 4 * (prev_free_slots - curr_free_slots))
&& i < 11);
+
xenvif_tx_dealloc_action(vif);
And worst case after 500 ns I let the thread to do the unmap anyway. But
I was a bit worried about this approach, so I choose a bit more
conservative one for this patch.
There are also ideas to use some other instrument for unmapping instead
of the current separate thread approach. Putting it into the NAPI
instance was the original idea, which caused problems. Placing it into
the another thread where RX work happens also doesn't sound too good,
these things can and should happen in parallel.
Other ideas were work queues and tasklets, I'll spend some more time to
check if they are feasible.
Regards,
Zoli
next prev parent reply other threads:[~2013-12-16 16:30 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-12 23:48 [PATCH net-next v2 0/9] xen-netback: TX grant mapping with SKBTX_DEV_ZEROCOPY instead of copy Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 1/9] xen-netback: Introduce TX grant map definitions Zoltan Kiss
2013-12-13 15:31 ` Wei Liu
2013-12-13 15:31 ` Wei Liu
2013-12-13 18:22 ` Zoltan Kiss
2013-12-13 18:22 ` Zoltan Kiss
2013-12-13 19:14 ` Wei Liu
2013-12-13 19:14 ` Wei Liu
2013-12-16 15:21 ` Zoltan Kiss
2013-12-16 15:21 ` Zoltan Kiss
2013-12-16 17:50 ` Wei Liu
2014-01-07 14:50 ` Zoltan Kiss
2014-01-07 14:50 ` Zoltan Kiss
2013-12-16 17:50 ` Wei Liu
2013-12-12 23:48 ` Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 2/9] xen-netback: Change TX path from grant copy to mapping Zoltan Kiss
2013-12-12 23:48 ` Zoltan Kiss
2013-12-13 15:36 ` Wei Liu
2013-12-13 15:36 ` Wei Liu
2013-12-16 15:38 ` Zoltan Kiss
2013-12-16 18:21 ` Wei Liu
2013-12-16 18:57 ` Zoltan Kiss
2013-12-16 19:06 ` Wei Liu
2013-12-16 19:06 ` Wei Liu
2013-12-16 18:57 ` Zoltan Kiss
2013-12-16 18:21 ` Wei Liu
2013-12-16 15:38 ` Zoltan Kiss
2013-12-17 21:49 ` [Xen-devel] " Konrad Rzeszutek Wilk
2013-12-30 17:58 ` Zoltan Kiss
2013-12-30 17:58 ` [Xen-devel] " Zoltan Kiss
2013-12-17 21:49 ` Konrad Rzeszutek Wilk
2013-12-12 23:48 ` [PATCH net-next v2 3/9] xen-netback: Remove old TX grant copy definitons and fix indentations Zoltan Kiss
2013-12-12 23:48 ` Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 4/9] xen-netback: Change RX path for mapped SKB fragments Zoltan Kiss
2013-12-12 23:48 ` Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 5/9] xen-netback: Add stat counters for zerocopy Zoltan Kiss
2013-12-12 23:48 ` Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 6/9] xen-netback: Handle guests with too many frags Zoltan Kiss
2013-12-12 23:48 ` Zoltan Kiss
2013-12-13 15:43 ` Wei Liu
2013-12-13 15:43 ` Wei Liu
2013-12-16 16:10 ` Zoltan Kiss
2013-12-16 16:10 ` Zoltan Kiss
2013-12-16 18:09 ` Wei Liu
2014-01-07 15:23 ` Zoltan Kiss
2014-01-07 15:23 ` Zoltan Kiss
2013-12-16 18:09 ` Wei Liu
2013-12-12 23:48 ` [PATCH net-next v2 7/9] xen-netback: Add stat counters for frag_list skbs Zoltan Kiss
2013-12-12 23:48 ` Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 8/9] xen-netback: Timeout packets in RX path Zoltan Kiss
2013-12-13 15:44 ` Wei Liu
2013-12-16 17:16 ` Zoltan Kiss
2013-12-16 19:03 ` Wei Liu
2013-12-16 19:03 ` Wei Liu
2013-12-16 17:16 ` Zoltan Kiss
2013-12-13 15:44 ` Wei Liu
2013-12-12 23:48 ` Zoltan Kiss
2013-12-12 23:48 ` [PATCH net-next v2 9/9] xen-netback: Aggregate TX unmap operations Zoltan Kiss
2013-12-13 15:44 ` Wei Liu
2013-12-13 15:44 ` Wei Liu
2013-12-16 16:30 ` Zoltan Kiss
2013-12-16 16:30 ` Zoltan Kiss [this message]
2013-12-12 23:48 ` Zoltan Kiss
2013-12-16 6:32 ` [PATCH net-next v2 0/9] xen-netback: TX grant mapping with SKBTX_DEV_ZEROCOPY instead of copy annie li
2013-12-16 6:32 ` [Xen-devel] " annie li
2013-12-16 16:13 ` Zoltan Kiss
2013-12-16 16:13 ` Zoltan Kiss
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52AF2A8A.50404@citrix.com \
--to=zoltan.kiss@citrix.com \
--cc=ian.campbell@citrix.com \
--cc=jonathan.davies@citrix.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.