All of lore.kernel.org
 help / color / mirror / Atom feed
From: Junichi Nomura <j-nomura@ce.jp.nec.com>
To: "Merla, ShivaKrishna" <ShivaKrishna.Merla@netapp.com>
Cc: "axboe@kernel.dk" <axboe@kernel.dk>,
	device-mapper development <dm-devel@redhat.com>,
	"jmoyer@redhat.com" <jmoyer@redhat.com>,
	Mike Snitzer <snitzer@redhat.com>
Subject: Re: [PATCH 6/8] dm: don't start current request if it would've merged with the previous
Date: Mon, 9 Mar 2015 06:09:56 +0000	[thread overview]
Message-ID: <54FD3934.5060700@ce.jp.nec.com> (raw)
In-Reply-To: <230f5fbd7d6c403ab81327a69a52361f@hioexcmbx01-prd.hq.netapp.com>

On 03/09/15 12:30, Merla, ShivaKrishna wrote:
>> Secondly, for this comment from Merla ShivaKrishna:
>>
>>> Yes, Indeed this the exact issue we saw at NetApp. While running sequential
>>> 4K write I/O with large thread count, 2 paths yield better performance than
>>> 4 paths and performance drastically drops with 4 paths. The device queue_depth
>>> as 32 and with blktrace we could see better I/O merging happening and average
>>> request size was > 8K through iostat. With 4 paths none of the I/O gets merged and
>>> always average request size is 4K. Scheduler used was noop as we are using SSD
>>> based storage. We could get I/O merging to happen even with 4 paths but with lower
>>> device queue_depth of 16. Even then the performance was lacking compared to 2 paths.
>>
>> Have you tried increasing nr_requests of the dm device?
>> E.g. setting nr_requests to 256.
>>
>> 4 paths with each queue depth 32 means that it can have 128 I/Os in flight.
>> With the default value of nr_requests 128, the request queue is almost
>> always empty and I/O merge could not happen.
>> Increasing nr_requests of the dm device allows some more requests
>> queued,
>> thus the chance of merging may increase.
>> Reducing the lower device queue depth could be another solution. But if
>> the depth is too low, you might not be able to keep the optimal speed.
>>
> Yes, we have tried this as well but didn't help. Indeed, we have tested with queue_depth
> of 16 on each path as well with 64 I/O's in flight and resulted in same issue. We did try
> reducing the queue_depth with 4 paths, but couldn't achieve comparable performance
> as of 2 paths. With Mike's patch, we see tremendous improvement with just a small delay 
> of ~20us with 4 paths. This might vary with different configurations but sure have proved 
> that a tunable to delay dispatches with sequential workloads has helped a lot.

Hi,

did you try increasing nr_requests of dm request queue?
If so, what was the increased value of nr_requests in the case of
device queue_depth 32?

-- 
Jun'ichi Nomura, NEC Corporation

  reply	other threads:[~2015-03-09  6:09 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-03-04  0:47 [PATCH 0/8] dm: optimize request-based queue processing Mike Snitzer
2015-03-04  0:47 ` [PATCH 1/8] dm: remove unnecessary wrapper around blk_lld_busy Mike Snitzer
2015-03-04  0:47 ` [PATCH 2/8] dm: remove request-based DM queue's lld_busy_fn hook Mike Snitzer
2015-03-04  0:47 ` [PATCH 3/8] dm: remove request-based logic from make_request_fn wrapper Mike Snitzer
2015-03-04  0:47 ` [PATCH 4/8] dm: only run the queue on completion if congested or no requests pending Mike Snitzer
2015-03-04  0:47 ` [PATCH 5/8] dm: don't schedule delayed run of the queue if nothing to do Mike Snitzer
2015-03-04  0:47 ` [PATCH 6/8] dm: don't start current request if it would've merged with the previous Mike Snitzer
2015-03-04  6:36   ` Hannes Reinecke
2015-03-04 17:26     ` Mike Snitzer
2015-03-09  1:04   ` Junichi Nomura
2015-03-09  3:30     ` Merla, ShivaKrishna
2015-03-09  6:09       ` Junichi Nomura [this message]
2015-03-09 16:10         ` Merla, ShivaKrishna
2015-03-10  1:05           ` Junichi Nomura
2015-03-10  1:59             ` Merla, ShivaKrishna
2015-03-10  5:43               ` Junichi Nomura
2015-03-04  0:47 ` [PATCH 7/8] dm sysfs: introduce ability to add writable attributes Mike Snitzer
2015-03-04  0:47 ` [PATCH 8/8] dm: impose configurable deadline for dm_request_fn's merge heuristic Mike Snitzer
2015-03-06 15:30   ` [PATCH 8/8 v2] " Mike Snitzer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=54FD3934.5060700@ce.jp.nec.com \
    --to=j-nomura@ce.jp.nec.com \
    --cc=ShivaKrishna.Merla@netapp.com \
    --cc=axboe@kernel.dk \
    --cc=dm-devel@redhat.com \
    --cc=jmoyer@redhat.com \
    --cc=snitzer@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.