From mboxrd@z Thu Jan 1 00:00:00 1970 From: annie li Subject: Re: Interesting observation with network event notification and batching Date: Mon, 17 Jun 2013 19:34:10 +0800 Message-ID: <51BEF432.4060804@oracle.com> References: <20130612101451.GF2765@zion.uk.xensource.com> <20130614185303.GC21280@phenom.dumpdata.com> <20130616095433.GA27462@zion.uk.xensource.com> <1371461913.3967.68.camel@zakaz.uk.xensource.com> <20130617103504.GC1757@zion.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20130617103504.GC1757@zion.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Wei Liu Cc: andrew.bennieston@citrix.com, Konrad Rzeszutek Wilk , xen-devel@lists.xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com List-Id: xen-devel@lists.xenproject.org On 2013-6-17 18:35, Wei Liu wrote: > On Mon, Jun 17, 2013 at 10:38:33AM +0100, Ian Campbell wrote: >> On Sun, 2013-06-16 at 10:54 +0100, Wei Liu wrote: >>>>> Konrad, IIRC you once mentioned you discovered something with event >>>>> notification, what's that? >>>> They were bizzare. I naively expected some form of # of physical NIC >>>> interrupts to be around the same as the VIF or less. And I figured >>>> that the amount of interrupts would be constant irregardless of the >>>> size of the packets. In other words #packets == #interrupts. >>>> >>> It could be that the frontend notifies the backend for every packet it >>> sends. This is not desirable and I don't expect the ring to behave that >>> way. >> It is probably worth checking that things are working how we think they >> should. i.e. that netback's calls to RING_FINAL_CHECK_FOR_.. and >> netfront's calls to RING_PUSH_..._AND_CHECK_NOTIFY are placed at >> suitable points to maximise batching. >> >> Is the RING_FINAL_CHECK_FOR_REQUESTS inside the xen_netbk_tx_build_gops >> loop right? This would push the req_event pointer to just after the last >> request, meaning the net request enqueued by the frontend would cause a >> notification -- even though the backend is actually still continuing to >> process requests and would have picked up that packet without further >> notification. n this case there is a fair bit of work left in the >> backend for this iteration i.e. plenty of opportunity for the frontend >> to queue more requests. >> >> The comments in ring.h say: >> * These macros will set the req_event/rsp_event field to trigger a >> * notification on the very next message that is enqueued. If you want to >> * create batches of work (i.e., only receive a notification after several >> * messages have been enqueued) then you will need to create a customised >> * version of the FINAL_CHECK macro in your own code, which sets the event >> * field appropriately. >> >> Perhaps we want to just use RING_HAS_UNCONSUMED_REQUESTS in that loop >> (and other similar loops) and add a FINAL check at the very end? >> >>>> But it was odd and I didn't go deeper in it to figure out what >>>> was happening. And also to figure out if for the VIF we could >>>> do something of #packets != #interrupts. And hopefully some >>>> mechanism to adjust so that the amount of interrupts would >>>> be lesser per packets (hand waving here). >>> I'm trying to do this now. >> What scheme do you have in mind? > Basically the one you mentioned above. > > Playing with various event pointers now. Did you collect data of how much requests netback processes when req_event is updated in RING_FINAL_CHECK_FOR_REQUESTS? I assume this value is pretty small from your test result. How about not updating req_event every time when there is no unconsumed request in RING_FINAL_CHECK_FOR_REQUESTS? Thanks Annie > > > Wei. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel