[PATCH 1/2] xen-blkback: implement safe iterator for the list of persistent grants

[PATCH 1/2] xen-blkback: implement safe iterator for the list of persistent grants

[Roger Pau Monne]

Change foreach_grant iterator to a safe version, that allows freeing
the element while iterating. Also move the free code in
free_persistent_gnts to prevent freeing the element before the rb_next
call.

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad@kernel.org>
Cc: xen-devel@lists.xen.org
---
 drivers/block/xen-blkback/blkback.c |   18 +++++++++++-------
 1 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/drivers/block/xen-blkback/blkback.c b/drivers/block/xen-blkback/blkback.c
index 74374fb..5ac841f 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -161,10 +161,12 @@ static int dispatch_rw_block_io(struct xen_blkif *blkif,
 static void make_response(struct xen_blkif *blkif, u64 id,
 			  unsigned short op, int st);
 
-#define foreach_grant(pos, rbtree, node) \
-	for ((pos) = container_of(rb_first((rbtree)), typeof(*(pos)), node); \
+#define foreach_grant_safe(pos, n, rbtree, node) \
+	for ((pos) = container_of(rb_first((rbtree)), typeof(*(pos)), node), \
+	     (n) = rb_next(&(pos)->node); \
 	     &(pos)->node != NULL; \
-	     (pos) = container_of(rb_next(&(pos)->node), typeof(*(pos)), node))
+	     (pos) = container_of(n, typeof(*(pos)), node), \
+	     (n) = (&(pos)->node != NULL) ? rb_next(&(pos)->node) : NULL)
 
 
 static void add_persistent_gnt(struct rb_root *root,
@@ -217,10 +219,11 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num)
 	struct gnttab_unmap_grant_ref unmap[BLKIF_MAX_SEGMENTS_PER_REQUEST];
 	struct page *pages[BLKIF_MAX_SEGMENTS_PER_REQUEST];
 	struct persistent_gnt *persistent_gnt;
+	struct rb_node *n;
 	int ret = 0;
 	int segs_to_unmap = 0;
 
-	foreach_grant(persistent_gnt, root, node) {
+	foreach_grant_safe(persistent_gnt, n, root, node) {
 		BUG_ON(persistent_gnt->handle ==
 			BLKBACK_INVALID_HANDLE);
 		gnttab_set_unmap_op(&unmap[segs_to_unmap],
@@ -230,9 +233,6 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num)
 			persistent_gnt->handle);
 
 		pages[segs_to_unmap] = persistent_gnt->page;
-		rb_erase(&persistent_gnt->node, root);
-		kfree(persistent_gnt);
-		num--;
 
 		if (++segs_to_unmap == BLKIF_MAX_SEGMENTS_PER_REQUEST ||
 			!rb_next(&persistent_gnt->node)) {
@@ -241,6 +241,10 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num)
 			BUG_ON(ret);
 			segs_to_unmap = 0;
 		}
+
+		rb_erase(&persistent_gnt->node, root);
+		kfree(persistent_gnt);
+		num--;
 	}
 	BUG_ON(num != 0);
 }
-- 
1.7.7.5 (Apple Git-26)

[PATCH 2/2] xen-blkfront: implement safe version of llist_for_each_entry

[Roger Pau Monne]

Implement a safe version of llist_for_each_entry, and use it in
blkif_free. Previously grants where freed while iterating the list,
which lead to dereferences when trying to fetch the next item.

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad@kernel.org>
Cc: xen-devel@lists.xen.org
---
 drivers/block/xen-blkfront.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 96e9b00..df21b05 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -143,6 +143,13 @@ static DEFINE_SPINLOCK(minor_lock);
 
 #define DEV_NAME	"xvd"	/* name in /dev */
 
+#define llist_for_each_entry_safe(pos, n, node, member)		\
+	for ((pos) = llist_entry((node), typeof(*(pos)), member),	\
+	     (n) = (pos)->member.next;					\
+	     &(pos)->member != NULL;					\
+	     (pos) = llist_entry(n, typeof(*(pos)), member),		\
+	     (n) = (&(pos)->member != NULL) ? (pos)->member.next : NULL)
+
 static int get_id_from_freelist(struct blkfront_info *info)
 {
 	unsigned long free = info->shadow_free;
@@ -792,6 +799,7 @@ static void blkif_free(struct blkfront_info *info, int suspend)
 {
 	struct llist_node *all_gnts;
 	struct grant *persistent_gnt;
+	struct llist_node *n;
 
 	/* Prevent new requests being issued until we fix things up. */
 	spin_lock_irq(&info->io_lock);
@@ -804,7 +812,7 @@ static void blkif_free(struct blkfront_info *info, int suspend)
 	/* Remove all persistent grants */
 	if (info->persistent_gnts_c) {
 		all_gnts = llist_del_all(&info->persistent_gnts);
-		llist_for_each_entry(persistent_gnt, all_gnts, node) {
+		llist_for_each_entry_safe(persistent_gnt, n, all_gnts, node) {
 			gnttab_end_foreign_access(persistent_gnt->gref, 0, 0UL);
 			__free_page(pfn_to_page(persistent_gnt->pfn));
 			kfree(persistent_gnt);
-- 
1.7.7.5 (Apple Git-26)

Re: [Xen-devel] [PATCH 2/2] xen-blkfront: implement safe version of llist_for_each_entry

[Konrad Rzeszutek Wilk]

On Tue, Dec 04, 2012 at 03:21:53PM +0100, Roger Pau Monne wrote:
> Implement a safe version of llist_for_each_entry, and use it in
> blkif_free. Previously grants where freed while iterating the list,
> which lead to dereferences when trying to fetch the next item.

Looks like xen-blkfront is the only user of this llist_for_each_entry.

Would it be more prudent to put the macro in the llist.h file?

> 
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Konrad Rzeszutek Wilk <konrad@kernel.org>
> Cc: xen-devel@lists.xen.org
> ---
>  drivers/block/xen-blkfront.c |   10 +++++++++-
>  1 files changed, 9 insertions(+), 1 deletions(-)
> 
> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
> index 96e9b00..df21b05 100644
> --- a/drivers/block/xen-blkfront.c
> +++ b/drivers/block/xen-blkfront.c
> @@ -143,6 +143,13 @@ static DEFINE_SPINLOCK(minor_lock);
>  
>  #define DEV_NAME	"xvd"	/* name in /dev */
>  
> +#define llist_for_each_entry_safe(pos, n, node, member)		\
> +	for ((pos) = llist_entry((node), typeof(*(pos)), member),	\
> +	     (n) = (pos)->member.next;					\
> +	     &(pos)->member != NULL;					\
> +	     (pos) = llist_entry(n, typeof(*(pos)), member),		\
> +	     (n) = (&(pos)->member != NULL) ? (pos)->member.next : NULL)
> +
>  static int get_id_from_freelist(struct blkfront_info *info)
>  {
>  	unsigned long free = info->shadow_free;
> @@ -792,6 +799,7 @@ static void blkif_free(struct blkfront_info *info, int suspend)
>  {
>  	struct llist_node *all_gnts;
>  	struct grant *persistent_gnt;
> +	struct llist_node *n;
>  
>  	/* Prevent new requests being issued until we fix things up. */
>  	spin_lock_irq(&info->io_lock);
> @@ -804,7 +812,7 @@ static void blkif_free(struct blkfront_info *info, int suspend)
>  	/* Remove all persistent grants */
>  	if (info->persistent_gnts_c) {
>  		all_gnts = llist_del_all(&info->persistent_gnts);
> -		llist_for_each_entry(persistent_gnt, all_gnts, node) {
> +		llist_for_each_entry_safe(persistent_gnt, n, all_gnts, node) {
>  			gnttab_end_foreign_access(persistent_gnt->gref, 0, 0UL);
>  			__free_page(pfn_to_page(persistent_gnt->pfn));
>  			kfree(persistent_gnt);
> -- 
> 1.7.7.5 (Apple Git-26)
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: [Xen-devel] [PATCH 2/2] xen-blkfront: implement safe version of llist_for_each_entry

[Roger Pau Monné]

On 07/12/12 21:20, Konrad Rzeszutek Wilk wrote:
> On Tue, Dec 04, 2012 at 03:21:53PM +0100, Roger Pau Monne wrote:
>> Implement a safe version of llist_for_each_entry, and use it in
>> blkif_free. Previously grants where freed while iterating the list,
>> which lead to dereferences when trying to fetch the next item.
> 
> Looks like xen-blkfront is the only user of this llist_for_each_entry.
> 
> Would it be more prudent to put the macro in the llist.h file?

I'm not able to find out who is the maintainer of llist, should I just
CC it's author?

Re: [Xen-devel] [PATCH 2/2] xen-blkfront: implement safe version of llist_for_each_entry

[Konrad Rzeszutek Wilk]

On Mon, Dec 10, 2012 at 01:15:50PM +0100, Roger Pau Monné wrote:
> On 07/12/12 21:20, Konrad Rzeszutek Wilk wrote:
> > On Tue, Dec 04, 2012 at 03:21:53PM +0100, Roger Pau Monne wrote:
> >> Implement a safe version of llist_for_each_entry, and use it in
> >> blkif_free. Previously grants where freed while iterating the list,
> >> which lead to dereferences when trying to fetch the next item.
> > 
> > Looks like xen-blkfront is the only user of this llist_for_each_entry.
> > 
> > Would it be more prudent to put the macro in the llist.h file?
> 
> I'm not able to find out who is the maintainer of llist, should I just
> CC it's author?

Sure. I CC-ed akpm here to solicit his input as well. Either way I am
OK wit this being in xen-blkfront but it just seems that it could
be useful in the llist file since that is where the non-safe version
resides.
> 

Re: [Xen-devel] [PATCH 2/2] xen-blkfront: implement safe version of llist_for_each_entry

[Roger Pau Monné]

On 10/12/12 16:15, Konrad Rzeszutek Wilk wrote:
> On Mon, Dec 10, 2012 at 01:15:50PM +0100, Roger Pau Monné wrote:
>> On 07/12/12 21:20, Konrad Rzeszutek Wilk wrote:
>>> On Tue, Dec 04, 2012 at 03:21:53PM +0100, Roger Pau Monne wrote:
>>>> Implement a safe version of llist_for_each_entry, and use it in
>>>> blkif_free. Previously grants where freed while iterating the list,
>>>> which lead to dereferences when trying to fetch the next item.
>>>
>>> Looks like xen-blkfront is the only user of this llist_for_each_entry.
>>>
>>> Would it be more prudent to put the macro in the llist.h file?
>>
>> I'm not able to find out who is the maintainer of llist, should I just
>> CC it's author?
> 
> Sure. I CC-ed akpm here to solicit his input as well. Either way I am
> OK wit this being in xen-blkfront but it just seems that it could
> be useful in the llist file since that is where the non-safe version
> resides.

I'm going to resend this series with llist_for_each_entry_safe added to
llist.h in a separated patch. I wouldn't like to delay this series
because we cannot get llist_for_each_entry_safe added to llist.h, that's
why I've added it to blkfront in the first place.

Re: [Xen-devel] [PATCH 2/2] xen-blkfront: implement safe version of llist_for_each_entry

[Andrew Morton]

On Mon, 10 Dec 2012 18:05:23 +0100
Roger Pau Monné <roger.pau@citrix.com> wrote:

> On 10/12/12 16:15, Konrad Rzeszutek Wilk wrote:
> > On Mon, Dec 10, 2012 at 01:15:50PM +0100, Roger Pau Monné wrote:
> >> On 07/12/12 21:20, Konrad Rzeszutek Wilk wrote:
> >>> On Tue, Dec 04, 2012 at 03:21:53PM +0100, Roger Pau Monne wrote:
> >>>> Implement a safe version of llist_for_each_entry, and use it in
> >>>> blkif_free. Previously grants where freed while iterating the list,
> >>>> which lead to dereferences when trying to fetch the next item.
> >>>
> >>> Looks like xen-blkfront is the only user of this llist_for_each_entry.
> >>>
> >>> Would it be more prudent to put the macro in the llist.h file?
> >>
> >> I'm not able to find out who is the maintainer of llist, should I just
> >> CC it's author?
> > 
> > Sure. I CC-ed akpm here to solicit his input as well. Either way I am
> > OK wit this being in xen-blkfront but it just seems that it could
> > be useful in the llist file since that is where the non-safe version
> > resides.
> 
> I'm going to resend this series with llist_for_each_entry_safe added to
> llist.h in a separated patch. I wouldn't like to delay this series
> because we cannot get llist_for_each_entry_safe added to llist.h, that's
> why I've added it to blkfront in the first place.

Please include that llist.h patch within the xen merge.  It's just a single
hunk and won't cause any disruption.