From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Bob Liu <bob.liu@oracle.com>
Cc: xen-devel@lists.xen.org, linux-kernel@vger.kernel.org,
roger.pau@citrix.com, felipe.franciosi@citrix.com, axboe@fb.com,
avanzini.arianna@gmail.com, rafal.mielniczuk@citrix.com,
jonathan.davies@citrix.com, david.vrabel@citrix.com
Subject: Re: [PATCH v4 04/10] xen/blkfront: split per device io_lock
Date: Tue, 3 Nov 2015 20:51:05 -0500 [thread overview]
Message-ID: <20151104015104.GA3882@x230.dumpdata.com> (raw)
In-Reply-To: <56395A40.70204@oracle.com>
On Wed, Nov 04, 2015 at 09:07:12AM +0800, Bob Liu wrote:
>
> On 11/04/2015 04:09 AM, Konrad Rzeszutek Wilk wrote:
> > On Mon, Nov 02, 2015 at 12:21:40PM +0800, Bob Liu wrote:
> >> The per device io_lock became a coarser grained lock after multi-queues/rings
> >> was introduced, this patch introduced a fine-grained ring_lock for each ring.
> >
> > s/was introduced/was introduced (see commit titled XYZ)/
> >
> > s/introdued/introduces/
> >>
> >> The old io_lock was renamed to dev_lock and only protect the ->grants list
> >
> > s/was/is/
> > s/protect/protects/
> >
> >> which is shared by all rings.
> >>
> >> Signed-off-by: Bob Liu <bob.liu@oracle.com>
> >> ---
> >> drivers/block/xen-blkfront.c | 57 ++++++++++++++++++++++++++------------------
> >> 1 file changed, 34 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
> >> index eab78e7..8cc5995 100644
> >> --- a/drivers/block/xen-blkfront.c
> >> +++ b/drivers/block/xen-blkfront.c
> >> @@ -121,6 +121,7 @@ MODULE_PARM_DESC(max_ring_page_order, "Maximum order of pages to be used for the
> >> */
> >> struct blkfront_ring_info {
> >> struct blkif_front_ring ring;
> >
> > Can you add a comment explaining the lock semantic? As in under what conditions
> > should it be taken? Like you have it below.
> >
> >> + spinlock_t ring_lock;
> >> unsigned int ring_ref[XENBUS_MAX_RING_PAGES];
> >> unsigned int evtchn, irq;
> >> struct work_struct work;
> >> @@ -138,7 +139,8 @@ struct blkfront_ring_info {
> >> */
> >> struct blkfront_info
> >> {
> >> - spinlock_t io_lock;
> >> + /* Lock to proect info->grants list shared by multi rings */
> >
> > s/proect/protect/
> >
> > Missing full stop.
> >
> >> + spinlock_t dev_lock;
> >
> > Shouldn't it be right next to what it is protecting?
> >
> > That is right below (or above): 'struct list_head grants;'?
> >
> >> struct mutex mutex;
> >> struct xenbus_device *xbdev;
> >> struct gendisk *gd;
> >> @@ -224,6 +226,7 @@ static int fill_grant_buffer(struct blkfront_ring_info *rinfo, int num)
> >> struct grant *gnt_list_entry, *n;
> >> int i = 0;
> >>
> >> + spin_lock_irq(&info->dev_lock);
> >
> > Why there? Why not where you add it to the list?
> >> while(i < num) {
> >> gnt_list_entry = kzalloc(sizeof(struct grant), GFP_NOIO);
> >> if (!gnt_list_entry)
> >> @@ -242,6 +245,7 @@ static int fill_grant_buffer(struct blkfront_ring_info *rinfo, int num)
> >> list_add(&gnt_list_entry->node, &info->grants);
> >
> > Right here that is?
> >
> > You are holding the lock for the duration of 'kzalloc' and 'alloc_page'.
> >
> > And more interestingly, GFP_NOIO translates to __GFP_WAIT which means
> > it can call 'schedule'. - And you have taken an spinlock. That should
> > have thrown lots of warnings?
> >
> >> i++;
> >> }
> >> + spin_unlock_irq(&info->dev_lock);
> >>
> >> return 0;
> >>
> >> @@ -254,6 +258,7 @@ out_of_memory:
> >> kfree(gnt_list_entry);
> >> i--;
> >> }
> >> + spin_unlock_irq(&info->dev_lock);
> >
> > Just do it around the 'list_del' operation. You are using an
> > 'safe'
> >> BUG_ON(i != 0);
> >> return -ENOMEM;
> >> }
> >> @@ -265,6 +270,7 @@ static struct grant *get_grant(grant_ref_t *gref_head,
> >> struct grant *gnt_list_entry;
> >> unsigned long buffer_gfn;
> >>
> >> + spin_lock(&info->dev_lock);
> >> BUG_ON(list_empty(&info->grants));
> >> gnt_list_entry = list_first_entry(&info->grants, struct grant,
> >> node);
> >> @@ -272,8 +278,10 @@ static struct grant *get_grant(grant_ref_t *gref_head,
> >>
> >> if (gnt_list_entry->gref != GRANT_INVALID_REF) {
> >> info->persistent_gnts_c--;
> >> + spin_unlock(&info->dev_lock);
> >> return gnt_list_entry;
> >> }
> >> + spin_unlock(&info->dev_lock);
> >
> > Just have one spin_unlock. Put it right before the 'if (gnt_list_entry->gref)..'.
>
> That's used to protect info->persistent_gnts_c, will update all other place.
But you don't mention that in the description - that the lock is suppose
to also protect persistent_gnts_c. Please update that.
>
> Thanks,
> -Bob
next prev parent reply other threads:[~2015-11-04 1:51 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-02 4:21 [PATCH v4 00/10] xen-block: multi hardware-queues/rings support Bob Liu
2015-11-02 4:21 ` [PATCH v4 01/10] xen/blkif: document blkif multi-queue/ring extension Bob Liu
2015-11-02 4:21 ` [PATCH v4 02/10] xen/blkfront: separate per ring information out of device info Bob Liu
2015-11-02 4:49 ` kbuild test robot
2015-11-02 5:33 ` Bob Liu
2015-11-02 4:21 ` [PATCH v4 03/10] xen/blkfront: pseudo support for multi hardware queues/rings Bob Liu
[not found] ` <20151103194436.GE28527@char.us.oracle.com>
2015-11-04 1:01 ` Bob Liu
2015-11-02 4:21 ` [PATCH v4 04/10] xen/blkfront: split per device io_lock Bob Liu
[not found] ` <20151103200902.GF28527@char.us.oracle.com>
2015-11-04 1:07 ` Bob Liu
2015-11-04 1:51 ` Konrad Rzeszutek Wilk [this message]
2015-11-02 4:21 ` [PATCH v4 05/10] xen/blkfront: negotiate number of queues/rings to be used with backend Bob Liu
[not found] ` <20151103204029.GH28527@char.us.oracle.com>
2015-11-04 1:11 ` Bob Liu
2015-11-04 1:53 ` Konrad Rzeszutek Wilk
2015-11-02 4:21 ` [PATCH v4 06/10] xen/blkback: separate ring information out of struct xen_blkif Bob Liu
2015-11-02 4:21 ` [PATCH v4 07/10] xen/blkback: pseudo support for multi hardware queues/rings Bob Liu
2015-11-05 2:30 ` Konrad Rzeszutek Wilk
2015-11-05 3:02 ` Bob Liu
2015-11-05 3:24 ` Konrad Rzeszutek Wilk
2015-11-02 4:21 ` [PATCH v4 08/10] xen/blkback: get the number of hardware queues/rings from blkfront Bob Liu
2015-11-05 2:37 ` Konrad Rzeszutek Wilk
2015-11-02 4:21 ` [PATCH v4 09/10] xen/blkfront: make persistent grants per-queue Bob Liu
2015-11-05 2:39 ` Konrad Rzeszutek Wilk
2015-11-02 4:21 ` [PATCH v4 10/10] xen/blkback: make pool of persistent grants and free pages per-queue Bob Liu
2015-11-05 2:43 ` Konrad Rzeszutek Wilk
2015-11-05 2:46 ` Bob Liu
2015-11-05 19:50 ` Konrad Rzeszutek Wilk
2015-11-02 11:19 ` [Xen-devel] [PATCH v4 00/10] xen-block: multi hardware-queues/rings support Julien Grall
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20151104015104.GA3882@x230.dumpdata.com \
--to=konrad.wilk@oracle.com \
--cc=avanzini.arianna@gmail.com \
--cc=axboe@fb.com \
--cc=bob.liu@oracle.com \
--cc=david.vrabel@citrix.com \
--cc=felipe.franciosi@citrix.com \
--cc=jonathan.davies@citrix.com \
--cc=linux-kernel@vger.kernel.org \
--cc=rafal.mielniczuk@citrix.com \
--cc=roger.pau@citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).