[PATCH 0/3] dm snapshot: Improve performance using a more fine-grained locking scheme

[Nikos Tsironis <ntsironis@arrikto.com>]

dm-snapshot uses a single mutex to serialize every access to the
snapshot state, including accesses to the exception hash tables. This
mutex is a bottleneck preventing dm-snapshot to scale as the number of
threads doing IO increases.

The major contention points are __origin_write()/snapshot_map() and
pending_complete(), i.e., the submission and completion of pending
exceptions.

This patchset substitutes the single mutex with:

  * A read-write semaphore, which protects the mostly read fields of the
    snapshot structure.

  * Per-bucket bit spinlocks, that protect accesses to the exception
    hash tables.

fio benchmarks using the null_blk device show significant performance
improvements as the number of worker processes increases. Write latency
is almost halved and write IOPS are nearly doubled.

The relevant patch provides detailed benchmark results.

A summary of the patchset follows:

  1. The first patch adds two helper functions to linux/list_bl.h, which
     is used to implement the per-bucket bit spinlocks in dm-snapshot.

  2. The second patch removes the need to sleep holding the snapshot
     lock in pending_complete(), thus allowing us to replace the mutex
     with the per-bucket bit spinlocks.

  3. The third patch changes the locking scheme, as described previously.

Nikos Tsironis (3):
  list_bl: Add hlist_bl_add_before/behind helpers
  dm snapshot: Don't sleep holding the snapshot lock
  dm snapshot: Use fine-grained locking scheme

 drivers/md/dm-exception-store.h |   3 +-
 drivers/md/dm-snap.c            | 359 +++++++++++++++++++++++++++-------------
 include/linux/list_bl.h         |  27 +++
 3 files changed, 269 insertions(+), 120 deletions(-)

-- 
2.11.0