linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/10] scsi: tracking device queue depth via sbitmap
@ 2020-02-11 12:11 Ming Lei
  2020-02-11 12:11 ` [PATCH 01/10] sbitmap: maintain allocation round_robin in sbitmap Ming Lei
                   ` (9 more replies)
  0 siblings, 10 replies; 15+ messages in thread
From: Ming Lei @ 2020-02-11 12:11 UTC (permalink / raw)
  To: James Bottomley, linux-scsi, Martin K . Petersen, linux-block,
	Jens Axboe
  Cc: Ming Lei, Omar Sandoval, Sathya Prakash, Chaitra P B,
	Suganath Prabu Subramani, Kashyap Desai, Sumit Saxena,
	Shivasharan S, Ewan D . Milne, Hannes Reinecke, Bart Van Assche

Hi,

scsi uses one global atomic variable to track queue depth for each
LUN/request queue. This way can't scale well when there is lots of CPU
cores and the disk is very fast. Broadcom guys has complained that their
high end HBA can't reach top performance because .device_busy is
operated in IO path.

Replace the atomic variable sdev->device_busy with sbitmap for
tracking scsi device queue depth.

Test on scsi_debug shows this way improve IOPS > 20%. Meantime
the IOPS difference is just ~1% compared with bypassing .device_busy
on scsi_debug via patches[1]

The 1st 6 patches moves percpu allocation hint into sbitmap, since
the improvement by doing percpu allocation hint on sbitmap is observable.
Meantime export helpers for SCSI.

Patch 7 and 8 prepares for the conversion by returning budget token
from .get_budget callback, meantime passes the budget token to driver
via 'struct blk_mq_queue_data' in .queue_rq().

The last two patches changes SCSI for switching to track device queue
depth via sbitmap.

Broadcom Guys, please test this patchset and see if expected performance
can be reached.

Please comment and review!

thanks,
Ming


[1] https://lore.kernel.org/linux-block/20200119071432.18558-6-ming.lei@redhat.com/

Ming Lei (10):
  sbitmap: maintain allocation round_robin in sbitmap
  sbitmap: add helpers for updating allocation hint
  sbitmap: remove sbitmap_clear_bit_unlock
  sbitmap: move allocation hint into sbitmap
  sbitmap: export sbitmap_weight
  sbitmap: add helper of sbitmap_calculate_shift
  blk-mq: return budget token from .get_budget callback
  blk-mq: pass budget token to dirver via blk_mq_queue_data
  scsi: add scsi_device_busy() to read sdev->device_busy
  scsi: replace sdev->device_busy with sbitmap

 block/blk-mq-sched.c                 |  20 ++-
 block/blk-mq.c                       |  37 +++--
 block/blk-mq.h                       |  11 +-
 block/kyber-iosched.c                |   3 +-
 drivers/dma/idxd/device.c            |   2 +-
 drivers/dma/idxd/submit.c            |   2 +-
 drivers/scsi/mpt3sas/mpt3sas_scsih.c |   2 +-
 drivers/scsi/scsi.c                  |   2 +
 drivers/scsi/scsi_lib.c              |  47 +++---
 drivers/scsi/scsi_priv.h             |   1 +
 drivers/scsi/scsi_scan.c             |  21 ++-
 drivers/scsi/scsi_sysfs.c            |   4 +-
 drivers/scsi/sg.c                    |   2 +-
 include/linux/blk-mq.h               |   5 +-
 include/linux/sbitmap.h              |  84 +++++++----
 include/scsi/scsi_cmnd.h             |   2 +
 include/scsi/scsi_device.h           |   8 +-
 lib/sbitmap.c                        | 213 +++++++++++++++------------
 18 files changed, 285 insertions(+), 181 deletions(-)

Cc: Omar Sandoval <osandov@fb.com>
Cc: Sathya Prakash <sathya.prakash@broadcom.com>
Cc: Chaitra P B <chaitra.basappa@broadcom.com>
Cc: Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>
Cc: Kashyap Desai <kashyap.desai@broadcom.com>
Cc: Sumit Saxena <sumit.saxena@broadcom.com>
Cc: Shivasharan S <shivasharan.srikanteshwara@broadcom.com>
Cc: Ewan D. Milne <emilne@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Bart Van Assche <bart.vanassche@wdc.com>
-- 
2.20.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2020-04-14  6:06 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-11 12:11 [PATCH 00/10] scsi: tracking device queue depth via sbitmap Ming Lei
2020-02-11 12:11 ` [PATCH 01/10] sbitmap: maintain allocation round_robin in sbitmap Ming Lei
2020-04-14  6:04   ` Hannes Reinecke
2020-02-11 12:11 ` [PATCH 02/10] sbitmap: add helpers for updating allocation hint Ming Lei
2020-04-14  6:05   ` Hannes Reinecke
2020-02-11 12:11 ` [PATCH 03/10] sbitmap: remove sbitmap_clear_bit_unlock Ming Lei
2020-04-14  6:06   ` Hannes Reinecke
2020-02-11 12:11 ` [PATCH 04/10] sbitmap: move allocation hint into sbitmap Ming Lei
2020-02-11 12:11 ` [PATCH 05/10] sbitmap: export sbitmap_weight Ming Lei
2020-02-11 12:11 ` [PATCH 06/10] sbitmap: add helper of sbitmap_calculate_shift Ming Lei
2020-02-11 12:11 ` [PATCH 07/10] blk-mq: return budget token from .get_budget callback Ming Lei
2020-02-11 12:11 ` [PATCH 08/10] blk-mq: pass budget token to dirver via blk_mq_queue_data Ming Lei
2020-02-11 12:11 ` [PATCH 09/10] scsi: add scsi_device_busy() to read sdev->device_busy Ming Lei
2020-02-11 12:11 ` [PATCH 10/10] scsi: replace sdev->device_busy with sbitmap Ming Lei
     [not found]   ` <202002140428.063yIjwM%lkp@intel.com>
2020-02-14  9:16     ` Ming Lei

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).