From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031942AbbKECnc (ORCPT ); Wed, 4 Nov 2015 21:43:32 -0500 Received: from userp1040.oracle.com ([156.151.31.81]:45321 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031689AbbKECnb (ORCPT ); Wed, 4 Nov 2015 21:43:31 -0500 Date: Wed, 4 Nov 2015 21:43:15 -0500 From: Konrad Rzeszutek Wilk To: Bob Liu Cc: xen-devel@lists.xen.org, linux-kernel@vger.kernel.org, roger.pau@citrix.com, felipe.franciosi@citrix.com, axboe@fb.com, avanzini.arianna@gmail.com, rafal.mielniczuk@citrix.com, jonathan.davies@citrix.com, david.vrabel@citrix.com Subject: Re: [PATCH v4 10/10] xen/blkback: make pool of persistent grants and free pages per-queue Message-ID: <20151105024314.GD3949@x230.dumpdata.com> References: <1446438106-20171-1-git-send-email-bob.liu@oracle.com> <1446438106-20171-11-git-send-email-bob.liu@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1446438106-20171-11-git-send-email-bob.liu@oracle.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Source-IP: aserv0022.oracle.com [141.146.126.234] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Nov 02, 2015 at 12:21:46PM +0800, Bob Liu wrote: > Make pool of persistent grants and free pages per-queue/ring instead of > per-device to get better scalability. How much better scalability do we get? .. snip .. > > > /* > - * pers_gnts_lock must be used around all the persistent grant helpers > - * because blkback may use multi-thread/queue for each backend. > + * We don't need locking around the persistent grant helpers > + * because blkback uses a single-thread for each backed, so we s/backed/backend/ > + * can be sure that this functions will never be called recursively. > + * > + * The only exception to that is put_persistent_grant, that can be called > + * from interrupt context (by xen_blkbk_unmap), so we have to use atomic > + * bit operations to modify the flags of a persistent grant and to count > + * the number of used grants. > */ ..snip.. > --- a/drivers/block/xen-blkback/common.h > +++ b/drivers/block/xen-blkback/common.h > @@ -282,6 +282,22 @@ struct xen_blkif_ring { > spinlock_t pending_free_lock; > wait_queue_head_t pending_free_wq; > > + /* buffer of free pages to map grant refs */ Full stop. > + spinlock_t free_pages_lock; > + int free_pages_num; > + struct list_head free_pages; > + > + /* tree to store persistent grants */ Full stop. > + spinlock_t pers_gnts_lock; > + struct rb_root persistent_gnts; > + unsigned int persistent_gnt_c; > + atomic_t persistent_gnt_in_use; > + unsigned long next_lru; > + > + /* used by the kworker that offload work from the persistent purge */ Full stop. > + struct list_head persistent_purge_list; > + struct work_struct persistent_purge_work; > + > /* statistics */ > unsigned long st_print; > unsigned long long st_rd_req;