All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mike Christie <michael.christie@oracle.com>
To: mst@redhat.com, stefanha@redhat.com, Chaitanya.Kulkarni@wdc.com,
	hch@lst.de, loberman@redhat.com, martin.petersen@oracle.com,
	linux-scsi@vger.kernel.org, target-devel@vger.kernel.org
Subject: PATCH 00/25 V4] target: fix cmd plugging and submission
Date: Fri, 12 Feb 2021 01:26:17 -0600	[thread overview]
Message-ID: <20210212072642.17520-1-michael.christie@oracle.com> (raw)

The following patches were made over Martin's 5.12 branches
to handle conflicts with the in_interrupt changes.

The patches fix the following issues:

1. target_core_iblock plugs and unplugs the queue for every
command. To handle this issue and handle an issue that
vhost-scsi and loop were avoiding by adding their own workqueue,
I added a new submission workqueue to LIO. Drivers can pass cmds
to it, and we can then submit batches of cmds.

2. vhost-scsi and loop on the submission side were doing a work
per cmd but because we can block in the block layer on resources
like tags we can end up creating lots of threads that will fight
each other. In this patchset I just use a cmd list per device to
avoid abusing the workueue layer and to better batch the cmds
to the lower layers.

The combined patchset fixes a major perf issue we've been hitting
with vhost-scsi where IOPs were stuck at 230K when running:

    fio --filename=/dev/sda  --direct=1 --rw=randrw --bs=4k
    --ioengine=libaio --iodepth=128  --numjobs=8 --time_based
    --group_reporting --runtime=60

The patches in this set get me to 350K when using devices that
have native IOPs of around 400-500K. 

3. Fix target_submit* error handling. While handling Christoph's
comment to kill target_submit_cmd_map_sgls I hit several bugs that
are now also fixed up.

V4:
- Fixed the target_submit error handling.
- Dropped get_cdb callback.
- Fixed kernel robot errors for incorrect return values and unused
variables.
- Used flush instead of cancel to fix bug in tmr code.
- Fixed race in tcmu.
- Made completion affinity handling a configfs setting
- Dropped patch that added the per device work lists. It really helped
a lot for higher perf initiators and tcm loop but only gave around a 5%
boost to other drivers. So I dropped it for now to see if there is
something more generic we can do.

V3:
- Fix rc type in target_submit so its a sense_reason_t
- Add BUG_ON if caller uses target_queue_cmd_submit but hasn't
implemented get_cdb.
- Drop unused variables in loop.
- Fix race in tcmu plug check
- Add comment about how plug check works in iblock
- Do a flush when handling TMRs instead of cancel

V2:
- Fix up container_of use coding style
- Handle offlist review comment from Laurence where with the
original code and my patches we can hit a bug where the cmd
times out, LIO starts up the TMR code, but it misses the cmd
because it's on the workqueue.
- Made the work per device work instead of session to handle
the previous issue and so if one dev hits some issue it sleeps on,
it won't block other devices.




             reply	other threads:[~2021-02-12  7:27 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-12  7:26 Mike Christie [this message]
2021-02-12  7:26 ` [PATCH 01/25] target: move t_task_cdb initialization Mike Christie
2021-02-12 19:07   ` Himanshu Madhani
2021-02-12  7:26 ` [PATCH 02/25] target: drop kref_get_unless_zero in target_get_sess_cmd Mike Christie
2021-02-12 19:08   ` Himanshu Madhani
2021-02-12  7:26 ` [PATCH 03/25] target: rename transport_init_se_cmd Mike Christie
2021-02-12 19:08   ` Himanshu Madhani
2021-02-12  7:26 ` [PATCH 04/25] target: break up target_submit_cmd_map_sgls Mike Christie
2021-02-12 19:09   ` Himanshu Madhani
2021-02-12  7:26 ` [PATCH 05/25] srpt: Convert to new submission API Mike Christie
2021-02-12 19:36   ` Bart Van Assche
2021-02-12  7:26 ` [PATCH 06/25] ibmvscsi_tgt: " Mike Christie
2021-02-12  7:26 ` [PATCH 07/25] qla2xxx: " Mike Christie
2021-02-12 19:16   ` Himanshu Madhani
2021-02-12  7:26 ` [PATCH 08/25] tcm_loop: " Mike Christie
2021-02-12  7:26 ` [PATCH 09/25] sbp_target: " Mike Christie
2021-02-12  7:26 ` [PATCH 10/25] usb gadget: " Mike Christie
2021-02-12  7:26 ` [PATCH 11/25] vhost-scsi: " Mike Christie
2021-02-16  8:40   ` Christoph Hellwig
2021-02-12  7:26 ` [PATCH 12/25] xen-scsiback: " Mike Christie
2021-02-12  7:26 ` [PATCH 13/25] tcm_fc: " Mike Christie
2021-02-12  7:26 ` [PATCH 14/25] target: remove target_submit_cmd_map_sgls Mike Christie
2021-02-12 22:17   ` Mike Christie
2021-02-16  8:40     ` Christoph Hellwig
2021-02-12  7:26 ` [PATCH 15/25] target: add gfp_t arg to target_cmd_init_cdb Mike Christie
2021-02-16  8:40   ` Christoph Hellwig
2021-02-12  7:26 ` [PATCH 16/25] target: add workqueue based cmd submission Mike Christie
2021-02-12  7:26 ` [PATCH 17/25] vhost scsi: use lio wq cmd submission helper Mike Christie
2021-02-12  7:26 ` [PATCH 18/25] tcm loop: use blk cmd allocator for se_cmds Mike Christie
2021-02-12  7:26 ` [PATCH 19/25] tcm loop: use lio wq cmd submission helper Mike Christie
2021-02-12  7:26 ` [PATCH 20/25] target: cleanup cmd flag bits Mike Christie
2021-02-12 19:17   ` Himanshu Madhani
2021-02-16  8:41   ` Christoph Hellwig
2021-02-12  7:26 ` [PATCH 21/25] target: fix backend plugging Mike Christie
2021-02-12  7:26 ` [PATCH 22/25] target iblock: add backend plug/unplug callouts Mike Christie
2021-02-12  7:26 ` [PATCH 23/25] target_core_user: " Mike Christie
2021-02-12  7:26 ` [PATCH 24/25] target: flush submission work during TMR processing Mike Christie
2021-02-12  7:26 ` [PATCH 25/25] target: make completion affinity configurable Mike Christie
2021-02-12 19:35   ` Himanshu Madhani
2021-02-15 16:32 ` PATCH 00/25 V4] target: fix cmd plugging and submission Laurence Oberman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210212072642.17520-1-michael.christie@oracle.com \
    --to=michael.christie@oracle.com \
    --cc=Chaitanya.Kulkarni@wdc.com \
    --cc=hch@lst.de \
    --cc=linux-scsi@vger.kernel.org \
    --cc=loberman@redhat.com \
    --cc=martin.petersen@oracle.com \
    --cc=mst@redhat.com \
    --cc=stefanha@redhat.com \
    --cc=target-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.