[PATCH 0/3] block: Per cpu merge stats and cfq fix

[PATCH 0/3] block: Per cpu merge stats and cfq fix

[Vivek Goyal]

Hi Jens,

Here is the patch for making merge stats per cpu. While scanning the code
I also noticed that in one case I was not freeing up per cpu stats. First
patch fixes it. I also included Namhyung Kim's bio merge fix.

Thanks
Vivek

Vivek Goyal (3):
  cfq-iosched: Fix a memory leak of per cpu stats for root group
  cfq-iosched: Make IO merge related stats per cpu
  block: call elv_bio_merged() when merged

 block/blk-cgroup.c  |   22 +++++++++++++++++-----
 block/blk-cgroup.h  |    4 ++--
 block/blk-core.c    |    2 ++
 block/cfq-iosched.c |    5 +++++
 4 files changed, 26 insertions(+), 7 deletions(-)

-- 
1.7.4.4

[PATCH 1/3] cfq-iosched: Fix a memory leak of per cpu stats for root group

[Vivek Goyal]

We allocated per cpu stats struct for root group but did not free it.
Fix it.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
---
 block/cfq-iosched.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index d646b27..0316f6e 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -3940,6 +3940,11 @@ static void cfq_exit_queue(struct elevator_queue *e)
 	 */
 	if (wait)
 		synchronize_rcu();
+
+#ifdef CONFIG_CFQ_GROUP_IOSCHED
+	/* Free up per cpu stats for root group */
+	free_percpu(cfqd->root_group.blkg.stats_cpu);
+#endif
 	kfree(cfqd);
 }
 
-- 
1.7.4.4

[PATCH 2/3] cfq-iosched: Make IO merge related stats per cpu

[Vivek Goyal]

Make BLKIO_STAT_MERGED per cpu hence gettring rid of need of taking
blkg->stats_lock.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
---
 block/blk-cgroup.c |   22 +++++++++++++++++-----
 block/blk-cgroup.h |    4 ++--
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index e41cc6f..07371cf 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -441,15 +441,27 @@ void blkiocg_update_completion_stats(struct blkio_group *blkg,
 }
 EXPORT_SYMBOL_GPL(blkiocg_update_completion_stats);
 
+/*  Merged stats are per cpu.  */
 void blkiocg_update_io_merged_stats(struct blkio_group *blkg, bool direction,
 					bool sync)
 {
+	struct blkio_group_stats_cpu *stats_cpu;
 	unsigned long flags;
 
-	spin_lock_irqsave(&blkg->stats_lock, flags);
-	blkio_add_stat(blkg->stats.stat_arr[BLKIO_STAT_MERGED], 1, direction,
-			sync);
-	spin_unlock_irqrestore(&blkg->stats_lock, flags);
+	/*
+	 * Disabling interrupts to provide mutual exclusion between two
+	 * writes on same cpu. It probably is not needed for 64bit. Not
+	 * optimizing that case yet.
+	 */
+	local_irq_save(flags);
+
+	stats_cpu = this_cpu_ptr(blkg->stats_cpu);
+
+	u64_stats_update_begin(&stats_cpu->syncp);
+	blkio_add_stat(stats_cpu->stat_arr_cpu[BLKIO_STAT_CPU_MERGED], 1,
+				direction, sync);
+	u64_stats_update_end(&stats_cpu->syncp);
+	local_irq_restore(flags);
 }
 EXPORT_SYMBOL_GPL(blkiocg_update_io_merged_stats);
 
@@ -1244,7 +1256,7 @@ static int blkiocg_file_read_map(struct cgroup *cgrp, struct cftype *cft,
 						BLKIO_STAT_WAIT_TIME, 1, 0);
 		case BLKIO_PROP_io_merged:
 			return blkio_read_blkg_stats(blkcg, cft, cb,
-						BLKIO_STAT_MERGED, 1, 0);
+						BLKIO_STAT_CPU_MERGED, 1, 1);
 		case BLKIO_PROP_io_queued:
 			return blkio_read_blkg_stats(blkcg, cft, cb,
 						BLKIO_STAT_QUEUED, 1, 0);
diff --git a/block/blk-cgroup.h b/block/blk-cgroup.h
index 2622267..a71d290 100644
--- a/block/blk-cgroup.h
+++ b/block/blk-cgroup.h
@@ -39,8 +39,6 @@ enum stat_type {
 	BLKIO_STAT_SERVICE_TIME = 0,
 	/* Total time spent waiting in scheduler queue in ns */
 	BLKIO_STAT_WAIT_TIME,
-	/* Number of IOs merged */
-	BLKIO_STAT_MERGED,
 	/* Number of IOs queued up */
 	BLKIO_STAT_QUEUED,
 	/* All the single valued stats go below this */
@@ -63,6 +61,8 @@ enum stat_type_cpu {
 	BLKIO_STAT_CPU_SERVICE_BYTES,
 	/* Total IOs serviced, post merge */
 	BLKIO_STAT_CPU_SERVICED,
+	/* Number of IOs merged */
+	BLKIO_STAT_CPU_MERGED,
 	BLKIO_STAT_CPU_NR
 };
 
-- 
1.7.4.4

[PATCH 3/3] block: call elv_bio_merged() when merged

[Vivek Goyal]

Commit 73c101011926 ("block: initial patch for on-stack per-task plugging")
removed calls to elv_bio_merged() when @bio merged with @req. Re-add them.

This in turn will update merged stats in associated group. That
should be safe as long as request has got reference to the blkio_group.

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Cc: Divyesh Shah <dpshah@google.com>
---
 block/blk-core.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 7369eee..c8303e9 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1122,6 +1122,7 @@ static bool bio_attempt_back_merge(struct request_queue *q, struct request *req,
 	req->ioprio = ioprio_best(req->ioprio, bio_prio(bio));
 
 	drive_stat_acct(req, 0);
+	elv_bio_merged(q, req, bio);
 	return true;
 }
 
@@ -1155,6 +1156,7 @@ static bool bio_attempt_front_merge(struct request_queue *q,
 	req->ioprio = ioprio_best(req->ioprio, bio_prio(bio));
 
 	drive_stat_acct(req, 0);
+	elv_bio_merged(q, req, bio);
 	return true;
 }
 
-- 
1.7.4.4

Re: [PATCH 0/3] block: Per cpu merge stats and cfq fix

[Vivek Goyal]

On Fri, May 20, 2011 at 08:02:44PM -0400, Vivek Goyal wrote:
> Hi Jens,
> 
> Here is the patch for making merge stats per cpu. While scanning the code
> I also noticed that in one case I was not freeing up per cpu stats. First
> patch fixes it. I also included Namhyung Kim's bio merge fix.
> 
> Thanks
> Vivek
> 
> Vivek Goyal (3):
>   cfq-iosched: Fix a memory leak of per cpu stats for root group
>   cfq-iosched: Make IO merge related stats per cpu

Minor correction. Following patch author is Namhyung Kim.

Namhyung Kim(1)

>   block: call elv_bio_merged() when merged

Thanks
Vivek

> 
>  block/blk-cgroup.c  |   22 +++++++++++++++++-----
>  block/blk-cgroup.h  |    4 ++--
>  block/blk-core.c    |    2 ++
>  block/cfq-iosched.c |    5 +++++
>  4 files changed, 26 insertions(+), 7 deletions(-)
> 
> -- 
> 1.7.4.4

Re: [PATCH 0/3] block: Per cpu merge stats and cfq fix

[Jens Axboe]

On 2011-05-21 02:02, Vivek Goyal wrote:
> Hi Jens,
> 
> Here is the patch for making merge stats per cpu. While scanning the code
> I also noticed that in one case I was not freeing up per cpu stats. First
> patch fixes it. I also included Namhyung Kim's bio merge fix.

Thanks, looks fine. Merged.

-- 
Jens Axboe

Re: [PATCH 0/3] block: Per cpu merge stats and cfq fix

[Jens Axboe]

On 2011-05-21 02:06, Vivek Goyal wrote:
> On Fri, May 20, 2011 at 08:02:44PM -0400, Vivek Goyal wrote:
>> Hi Jens,
>>
>> Here is the patch for making merge stats per cpu. While scanning the code
>> I also noticed that in one case I was not freeing up per cpu stats. First
>> patch fixes it. I also included Namhyung Kim's bio merge fix.
>>
>> Thanks
>> Vivek
>>
>> Vivek Goyal (3):
>>   cfq-iosched: Fix a memory leak of per cpu stats for root group
>>   cfq-iosched: Make IO merge related stats per cpu
> 
> Minor correction. Following patch author is Namhyung Kim.
> 
> Namhyung Kim(1)

Ugh, already merged it. I should have noticed that too, sorry about that
Namhyung.

-- 
Jens Axboe

Re: [PATCH 0/3] block: Per cpu merge stats and cfq fix

[Namhyung Kim]

2011-05-20 (금), 20:06 -0400, Vivek Goyal:
> On Fri, May 20, 2011 at 08:02:44PM -0400, Vivek Goyal wrote:
> > Hi Jens,
> > 
> > Here is the patch for making merge stats per cpu. While scanning the code
> > I also noticed that in one case I was not freeing up per cpu stats. First
> > patch fixes it. I also included Namhyung Kim's bio merge fix.
> > 
> > Thanks
> > Vivek
> > 
> > Vivek Goyal (3):
> >   cfq-iosched: Fix a memory leak of per cpu stats for root group
> >   cfq-iosched: Make IO merge related stats per cpu
> 
> Minor correction. Following patch author is Namhyung Kim.
> 
> Namhyung Kim(1)
> 
> >   block: call elv_bio_merged() when merged
> 
> Thanks
> Vivek
> 

Hi, Vivek. Thanks a lot for your work. :)


-- 
Regards,
Namhyung Kim

Re: [PATCH 0/3] block: Per cpu merge stats and cfq fix

[Namhyung Kim]

2011-05-23 (월), 10:04 +0200, Jens Axboe:
> On 2011-05-21 02:06, Vivek Goyal wrote:
> > On Fri, May 20, 2011 at 08:02:44PM -0400, Vivek Goyal wrote:
> >> Hi Jens,
> >>
> >> Here is the patch for making merge stats per cpu. While scanning the code
> >> I also noticed that in one case I was not freeing up per cpu stats. First
> >> patch fixes it. I also included Namhyung Kim's bio merge fix.
> >>
> >> Thanks
> >> Vivek
> >>
> >> Vivek Goyal (3):
> >>   cfq-iosched: Fix a memory leak of per cpu stats for root group
> >>   cfq-iosched: Make IO merge related stats per cpu
> > 
> > Minor correction. Following patch author is Namhyung Kim.
> > 
> > Namhyung Kim(1)
> 
> Ugh, already merged it. I should have noticed that too, sorry about that
> Namhyung.
> 

Hi, Jens.
It's all right, I'm fine :)


-- 
Regards,
Namhyung Kim