All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yipeng Wang <yipeng1.wang@intel.com>
To: pablo.de.lara.guarch@intel.com
Cc: dev@dpdk.org, yipeng1.wang@intel.com, bruce.richardson@intel.com,
	honnappa.nagarahalli@arm.com, vguvva@caviumnetworks.com,
	brijesh.s.singh@gmail.com
Subject: [PATCH v2 0/6] Add read-write concurrency to rte_hash library
Date: Fri, 29 Jun 2018 05:24:51 -0700	[thread overview]
Message-ID: <1530275097-123488-1-git-send-email-yipeng1.wang@intel.com> (raw)
In-Reply-To: <1528455078-328182-1-git-send-email-yipeng1.wang@intel.com>

This patch set adds the read-write concurrency support in rte_hash.
A new flag value is added to indicate if read-write concurrency is needed
during creation time. Test cases are implemented to do functional and
performance tests.

The new concurrency model is based on rte_rwlock. When Intel TSX is
available and the users indicate to use it, the TM version of the
rte_rwlock will be called. Both multi-writer and read-write concurrency
are protected by the rte_rwlock instead of the x86 specific RTM
instructions, so the x86 specific header rte_cuckoo_hash_x86.h is removed
and the code is infused into the main .c file.

A new rte_hash_count API is proposed to count how many keys are inserted
into the hash table.

v2->v1:
1. Split each commit into two commits for easier review (Pablo).
2. Add more comments in various places (Pablo).
3. hash: In key insertion function, move duplicated key checking to
earlier location and protect it using locks. Checking duplicated key
should happen first and data updates should be protected.
4. hash: In lookup bulk function, put signature comparison in lock,
since writers could happen between signature match on two buckets.
5. hash: Add write locks to reset function as well to protect resets.
5. test: Fix 32-bit compilation error in read-write test (Pablo).
6. test: Check total physical core count in read-write test. Don't
test with thread count that larger than physical core count.
7. Other minor fixes such as typos (Pablo).

Yipeng Wang (6):
  hash: make duplicated code into functions
  hash: add read and write concurrency support
  test: add tests in hash table perf test
  test: add test case for read write concurrency
  hash: fix to have more accurate key slot size
  hash: add new API function to query the key count

 lib/librte_hash/meson.build           |   1 -
 lib/librte_hash/rte_cuckoo_hash.c     | 695 +++++++++++++++++++++-------------
 lib/librte_hash/rte_cuckoo_hash.h     |  18 +-
 lib/librte_hash/rte_cuckoo_hash_x86.h | 164 --------
 lib/librte_hash/rte_hash.h            |  14 +
 lib/librte_hash/rte_hash_version.map  |   8 +
 test/test/Makefile                    |   1 +
 test/test/test_hash.c                 |  12 +
 test/test/test_hash_multiwriter.c     |   9 +
 test/test/test_hash_perf.c            |  36 +-
 test/test/test_hash_readwrite.c       | 645 +++++++++++++++++++++++++++++++
 11 files changed, 1155 insertions(+), 448 deletions(-)
 delete mode 100644 lib/librte_hash/rte_cuckoo_hash_x86.h
 create mode 100644 test/test/test_hash_readwrite.c

-- 
2.7.4

  parent reply	other threads:[~2018-06-29 19:31 UTC|newest]

Thread overview: 65+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-08 10:51 [PATCH v1 0/3] Add read-write concurrency to rte_hash library Yipeng Wang
2018-06-08 10:51 ` [PATCH v1 1/3] hash: add read and write concurrency support Yipeng Wang
2018-06-26 14:59   ` De Lara Guarch, Pablo
2018-06-08 10:51 ` [PATCH v1 2/3] test: add test case for read write concurrency Yipeng Wang
2018-06-26 15:48   ` De Lara Guarch, Pablo
2018-06-08 10:51 ` [PATCH v1 3/3] hash: add new API function to query the key count Yipeng Wang
2018-06-26 16:11   ` De Lara Guarch, Pablo
2018-06-29 12:24 ` Yipeng Wang [this message]
2018-06-29 12:24   ` [PATCH v2 1/6] hash: make duplicated code into functions Yipeng Wang
2018-07-06 10:04     ` De Lara Guarch, Pablo
2018-06-29 12:24   ` [PATCH v2 2/6] hash: add read and write concurrency support Yipeng Wang
2018-07-06 17:11     ` De Lara Guarch, Pablo
2018-06-29 12:24   ` [PATCH v2 3/6] test: add tests in hash table perf test Yipeng Wang
2018-07-06 17:17     ` De Lara Guarch, Pablo
2018-06-29 12:24   ` [PATCH v2 4/6] test: add test case for read write concurrency Yipeng Wang
2018-07-06 17:31     ` De Lara Guarch, Pablo
2018-06-29 12:24   ` [PATCH v2 5/6] hash: fix to have more accurate key slot size Yipeng Wang
2018-07-06 17:32     ` De Lara Guarch, Pablo
2018-06-29 12:24   ` [PATCH v2 6/6] hash: add new API function to query the key count Yipeng Wang
2018-07-06 17:36     ` De Lara Guarch, Pablo
2018-07-06 19:46 ` [PATCH v3 0/8] Add read-write concurrency to rte_hash library Yipeng Wang
2018-07-06 19:46   ` [PATCH v3 1/8] hash: fix multiwriter lock memory allocation Yipeng Wang
2018-07-09 11:26     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 2/8] hash: fix a multi-writer bug Yipeng Wang
2018-07-09 14:16     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 3/8] hash: fix to have more accurate key slot size Yipeng Wang
2018-07-09 14:20     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 4/8] hash: make duplicated code into functions Yipeng Wang
2018-07-09 14:25     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 5/8] hash: add read and write concurrency support Yipeng Wang
2018-07-09 14:28     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 6/8] test: add tests in hash table perf test Yipeng Wang
2018-07-09 15:33     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 7/8] test: add test case for read write concurrency Yipeng Wang
2018-07-09 16:24     ` De Lara Guarch, Pablo
2018-07-06 19:46   ` [PATCH v3 8/8] hash: add new API function to query the key count Yipeng Wang
2018-07-09 16:22     ` De Lara Guarch, Pablo
2018-07-09 10:44 ` [PATCH v4 0/8] Add read-write concurrency to rte_hash library Yipeng Wang
2018-07-09 10:44   ` [PATCH v4 1/8] hash: fix multiwriter lock memory allocation Yipeng Wang
2018-07-09 10:44   ` [PATCH v4 2/8] hash: fix a multi-writer race condition Yipeng Wang
2018-07-09 10:44   ` [PATCH v4 3/8] hash: fix key slot size accuracy Yipeng Wang
2018-07-09 10:44   ` [PATCH v4 4/8] hash: make duplicated code into functions Yipeng Wang
2018-07-09 10:45   ` [PATCH v4 5/8] hash: add read and write concurrency support Yipeng Wang
2018-07-09 10:45   ` [PATCH v4 6/8] test: add tests in hash table perf test Yipeng Wang
2018-07-09 10:45   ` [PATCH v4 7/8] test: add test case for read write concurrency Yipeng Wang
2018-07-09 10:45   ` [PATCH v4 8/8] hash: add new API function to query the key count Yipeng Wang
2018-07-10 18:00   ` [PATCH v4 0/8] Add read-write concurrency to rte_hash library Honnappa Nagarahalli
2018-07-12  1:31     ` Wang, Yipeng1
2018-07-12  2:36       ` Honnappa Nagarahalli
2018-07-13  1:47         ` Wang, Yipeng1
2018-07-10 16:59 ` [PATCH v5 " Yipeng Wang
2018-07-10 16:59   ` [PATCH v5 1/8] hash: fix multiwriter lock memory allocation Yipeng Wang
2018-07-10 16:59   ` [PATCH v5 2/8] hash: fix a multi-writer race condition Yipeng Wang
2018-07-10 16:59   ` [PATCH v5 3/8] hash: fix key slot size accuracy Yipeng Wang
2018-07-10 16:59   ` [PATCH v5 4/8] hash: make duplicated code into functions Yipeng Wang
2018-07-10 16:59   ` [PATCH v5 5/8] hash: add read and write concurrency support Yipeng Wang
2018-07-11 20:49     ` Stephen Hemminger
2018-07-12  1:22       ` Wang, Yipeng1
2018-07-12 20:30         ` Thomas Monjalon
2018-07-13  1:55           ` Wang, Yipeng1
2018-08-17 12:51             ` ASM
2018-07-10 16:59   ` [PATCH v5 6/8] test: add tests in hash table perf test Yipeng Wang
2018-07-10 17:00   ` [PATCH v5 7/8] test: add test case for read write concurrency Yipeng Wang
2018-07-10 17:00   ` [PATCH v5 8/8] hash: add new API function to query the key count Yipeng Wang
2018-07-12 21:03   ` [PATCH v5 0/8] Add read-write concurrency to rte_hash library Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1530275097-123488-1-git-send-email-yipeng1.wang@intel.com \
    --to=yipeng1.wang@intel.com \
    --cc=brijesh.s.singh@gmail.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=honnappa.nagarahalli@arm.com \
    --cc=pablo.de.lara.guarch@intel.com \
    --cc=vguvva@caviumnetworks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.