All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mike Snitzer <snitzer@redhat.com>
To: Mikulas Patocka <mpatocka@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
	dm-devel@redhat.com, linux-block@vger.kernel.org,
	"Alasdair G. Kergon" <agk@redhat.com>,
	Christoph Hellwig <hch@infradead.org>
Subject: Re: [PATCH 0/3] per-cpu in_flight counters for bio-based drivers
Date: Fri, 30 Nov 2018 14:57:31 -0500	[thread overview]
Message-ID: <20181130195731.GB3719@redhat.com> (raw)
In-Reply-To: <20181130155054.GA13744@redhat.com>

On Fri, Nov 30 2018 at 10:50am -0500,
Mike Snitzer <snitzer@redhat.com> wrote:

> On Fri, Nov 30 2018 at  9:43am -0500,
> Mike Snitzer <snitzer@redhat.com> wrote:
> 
> > On Tue, Nov 27 2018 at  7:42pm -0500,
> > Mikulas Patocka <mpatocka@redhat.com> wrote:
> > 
> > > These are the patches for per-cpu in_flight counters.
> > 
> > Do you have updated before vs after performance results for these
> > changes?
> > 
> > I'd imagine they are comparable to your previous run (though that run
> > included some other DM changes that I already staged).
> 
> Would like to see before vs after with:
> 
> http://git.kernel.dk/cgit/linux-block/log/?h=for-4.21/block
> vs
> https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=block-dm-4.21-inflight
> 
> (block-dm-4.21-inflight is based on latest for-4.21/block -- and it
> contains DM changes in front of the block changes from this thread; I'd
> prefer Jens take the changes like this rather than leave DM in a bit of
> a mess for me/us to have to cleanup later)

I ran the same fio test you did (in the previous thread where you did
the switch to percpu local to DM, rather than in block) _except_ I used
a ramdisk-based pmem device rather than a pure ramdisk:

fio --ioengine=psync --iodepth=1 --rw=read --bs=512 --direct=1 --numjobs=12 --time_based --runtime=10 --group_reporting --name=/dev/pmem0

2 6-core processors (w/ HT, so 24 logical cpus):

/dev/pmem0                                        14.6M
/dev/pmem0 with percpu counters                   14.8M
/dev/mapper/linear                                4736k
/dev/mapper/linear with percpu counters           4595k

It is only after I apply this commit that I can realize a big
performance win:
https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=block-dm-4.21-inflight&id=335b41c7513110b1519d8a93d412c138bf671263

/dev/mapper/linear with percpu + pending removed  11.2M

WARNING: multiple messages have this Message-ID (diff)
From: Mike Snitzer <snitzer@redhat.com>
To: Mikulas Patocka <mpatocka@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
	linux-block@vger.kernel.org, dm-devel@redhat.com,
	"Alasdair G. Kergon" <agk@redhat.com>,
	Christoph Hellwig <hch@infradead.org>
Subject: Re: [PATCH 0/3] per-cpu in_flight counters for bio-based drivers
Date: Fri, 30 Nov 2018 14:57:31 -0500	[thread overview]
Message-ID: <20181130195731.GB3719@redhat.com> (raw)
In-Reply-To: <20181130155054.GA13744@redhat.com>

On Fri, Nov 30 2018 at 10:50am -0500,
Mike Snitzer <snitzer@redhat.com> wrote:

> On Fri, Nov 30 2018 at  9:43am -0500,
> Mike Snitzer <snitzer@redhat.com> wrote:
> 
> > On Tue, Nov 27 2018 at  7:42pm -0500,
> > Mikulas Patocka <mpatocka@redhat.com> wrote:
> > 
> > > These are the patches for per-cpu in_flight counters.
> > 
> > Do you have updated before vs after performance results for these
> > changes?
> > 
> > I'd imagine they are comparable to your previous run (though that run
> > included some other DM changes that I already staged).
> 
> Would like to see before vs after with:
> 
> http://git.kernel.dk/cgit/linux-block/log/?h=for-4.21/block
> vs
> https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=block-dm-4.21-inflight
> 
> (block-dm-4.21-inflight is based on latest for-4.21/block -- and it
> contains DM changes in front of the block changes from this thread; I'd
> prefer Jens take the changes like this rather than leave DM in a bit of
> a mess for me/us to have to cleanup later)

I ran the same fio test you did (in the previous thread where you did
the switch to percpu local to DM, rather than in block) _except_ I used
a ramdisk-based pmem device rather than a pure ramdisk:

fio --ioengine=psync --iodepth=1 --rw=read --bs=512 --direct=1 --numjobs=12 --time_based --runtime=10 --group_reporting --name=/dev/pmem0

2 6-core processors (w/ HT, so 24 logical cpus):

/dev/pmem0                                        14.6M
/dev/pmem0 with percpu counters                   14.8M
/dev/mapper/linear                                4736k
/dev/mapper/linear with percpu counters           4595k

It is only after I apply this commit that I can realize a big
performance win:
https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=block-dm-4.21-inflight&id=335b41c7513110b1519d8a93d412c138bf671263

/dev/mapper/linear with percpu + pending removed  11.2M

  reply	other threads:[~2018-11-30 19:57 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-28  0:42 [PATCH 0/3] per-cpu in_flight counters for bio-based drivers Mikulas Patocka
2018-11-28  0:42 ` Mikulas Patocka
2018-11-30 14:43 ` Mike Snitzer
2018-11-30 14:43   ` Mike Snitzer
2018-11-30 15:50   ` Mike Snitzer
2018-11-30 15:50     ` Mike Snitzer
2018-11-30 19:57     ` Mike Snitzer [this message]
2018-11-30 19:57       ` Mike Snitzer
2018-11-30 22:22 ` [PATCH v2 0/6] " Mike Snitzer
2018-11-30 22:22   ` Mike Snitzer
2018-11-30 22:22   ` [PATCH v2 1/6] dm: dont rewrite dm_disk(md)->part0.in_flight Mike Snitzer
2018-11-30 22:22     ` Mike Snitzer
2018-11-30 22:22   ` [PATCH v2 2/6] dm rq: leverage blk_mq_queue_busy() to check for outstanding IO Mike Snitzer
2018-11-30 22:22     ` Mike Snitzer
2018-11-30 22:22   ` [PATCH v2 3/6] block: delete part_round_stats and switch to less precise counting Mike Snitzer
2018-11-30 22:22     ` Mike Snitzer
2018-11-30 22:22   ` [PATCH v2 4/6] block: switch to per-cpu in-flight counters Mike Snitzer
2018-11-30 22:22     ` Mike Snitzer
2018-12-05 17:30     ` Jens Axboe
2018-12-05 17:30       ` Jens Axboe
2018-12-05 17:49       ` Mike Snitzer
2018-12-05 17:49         ` Mike Snitzer
2018-12-05 17:54         ` Jens Axboe
2018-12-05 17:54           ` Jens Axboe
2018-12-05 18:03           ` Mike Snitzer
2018-12-05 18:03             ` Mike Snitzer
2018-12-05 18:04             ` Jens Axboe
2018-12-05 18:04               ` Jens Axboe
2018-12-05 18:18               ` Mike Snitzer
2018-12-05 18:18                 ` Mike Snitzer
2018-12-05 18:35                 ` Jens Axboe
2018-12-05 18:35                   ` Jens Axboe
2018-11-30 22:22   ` [PATCH v2 5/6] block: return just one value from part_in_flight Mike Snitzer
2018-11-30 22:22     ` Mike Snitzer
2018-11-30 22:22   ` [PATCH v2 6/6] dm: remove the pending IO accounting Mike Snitzer
2018-11-30 22:22     ` Mike Snitzer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181130195731.GB3719@redhat.com \
    --to=snitzer@redhat.com \
    --cc=agk@redhat.com \
    --cc=axboe@kernel.dk \
    --cc=dm-devel@redhat.com \
    --cc=hch@infradead.org \
    --cc=linux-block@vger.kernel.org \
    --cc=mpatocka@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.