All of lore.kernel.org
 help / color / mirror / Atom feed
From: Matthew Wilcox <willy@linux.intel.com>
To: linux-kernel@vger.kernel.org, Andrew Morton <akpm@linux-foundation.org>
Cc: Matthew Wilcox <willy@linux.intel.com>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	Konstantin Khlebnikov <koct9i@gmail.com>,
	Kirill Shutemov <kirill.shutemov@linux.intel.com>,
	Jan Kara <jack@suse.com>, Neil Brown <neilb@suse.de>,
	Ross Zwisler <ross.zwisler@linux.intel.com>
Subject: [PATCH v2 00/29] Radix tree multiorder fixes
Date: Thu, 14 Apr 2016 10:16:21 -0400	[thread overview]
Message-ID: <1460643410-30196-1-git-send-email-willy@linux.intel.com> (raw)

I must apologise for commit f96d18ff84 which left the impression that
the support for multiorder radix tree entries was functional.  As soon as
Ross tried to use it, it became apparent that my testing was completely
inadequate, and it didn't even work a little bit for orders that were
not a multiple of shift.

This series of patches is the result of about 6 weeks of redesign,
reimplementation, testing, arguing and hair-pulling.  The great news is
that the test-suite is now far better than it was.  That's reflected in
the diffstat for the test-suite alone:
 12 files changed, 436 insertions(+), 28 deletions(-)

The highlight for users of the tree is that the restriction on the order
of inserted entries being >= RADIX_TREE_MAP_SHIFT is now gone; the radix
tree now supports any order between 0 and 64.

For those who are interested in how the tree works, patch 9 is probably
the most interesting one as it introduces the new machinery for handling
sibling entries.

I've tried to be fair in attributing authorship to the person who
contributed the majority of the code in each patch; Ross has been an
invaluable partner in the development of this support and it's fair to
say that each of us has code in every commit.

I should also express my appreciation of the 0day testing.  It prompted
me that I was bloating the tinyconfig in an unacceptable way, and it
bisected to a commit which contained a rather nasty memory-corruption bug.

Changes since v1:

 - Rename get_sibling_offset() to get_slot_offset()
 - Use get_slot_offset() in radix_tree_insert() instead of doing arithmetic
 - Use get_slot_offset() in radix_tree_descend()
 - Some whitespace fixes
 - Fix race in radix_tree_next_chunk() against a multiorder insertion
 - Reduce the nuber of ifdefs by introducing iter_shift()
 - Fix some minor bugs in radix_tree_dump()
 - Merge the commit to fix shrinking bugs with the commit to add a test case
   for the shrinking code
 - Fix a bug in radix_tree_range_tag_if_tagged() if the first index was an
   alias of a multiorder entry
 - Add more test-cases to the test suite
 - Make __radix_tree_lookup fit the same pattern as the other tree walkers

Matthew Wilcox (18):
  radix-tree: Introduce radix_tree_empty
  radix tree test suite: Fix build
  radix tree test suite: Add tests for radix_tree_locate_item()
  Introduce CONFIG_RADIX_TREE_MULTIORDER
  radix-tree: Add missing sibling entry functionality
  radix-tree: Fix sibling entry insertion
  radix-tree: Fix deleting a multi-order entry through an alias
  radix-tree: Remove restriction on multi-order entries
  radix-tree: Introduce radix_tree_load_root()
  radix-tree: Fix extending the tree for multi-order entries at offset 0
  radix tree test suite: Start adding multiorder tests
  radix-tree: Fix several shrinking bugs with multiorder entries
  radix-tree: Rewrite __radix_tree_lookup
  radix-tree: Fix multiorder BUG_ON in radix_tree_insert
  radix-tree: Fix radix_tree_create for sibling entries
  radix-tree: Rewrite radix_tree_locate_item
  radix-tree: Fix radix_tree_range_tag_if_tagged() for multiorder
    entries
  radix-tree: Add copyright statements

Ross Zwisler (11):
  radix tree test suite: Allow testing other fan-out values
  radix tree test suite: keep regression test runs short
  radix tree test suite: rebuild when headers change
  radix-tree: remove unused looping macros
  radix-tree: add support for multi-order iterating
  radix tree test suite: multi-order iteration test
  radix-tree: Rewrite radix_tree_tag_set
  radix-tree: Rewrite radix_tree_tag_clear
  radix-tree: Rewrite radix_tree_tag_get
  radix-tree test suite: add multi-order tag test
  radix-tree: Fix radix_tree_dump() for multi-order entries

 include/linux/radix-tree.h                    | 106 +++--
 kernel/irq/irqdomain.c                        |   7 +-
 lib/Kconfig                                   |   3 +
 lib/radix-tree.c                              | 611 ++++++++++++++------------
 mm/Kconfig                                    |   1 +
 tools/testing/radix-tree/Makefile             |   4 +-
 tools/testing/radix-tree/generated/autoconf.h |   3 +
 tools/testing/radix-tree/linux/init.h         |   0
 tools/testing/radix-tree/linux/kernel.h       |  15 +-
 tools/testing/radix-tree/linux/slab.h         |   1 -
 tools/testing/radix-tree/linux/types.h        |   7 +-
 tools/testing/radix-tree/main.c               |  71 ++-
 tools/testing/radix-tree/multiorder.c         | 326 ++++++++++++++
 tools/testing/radix-tree/regression2.c        |   7 -
 tools/testing/radix-tree/tag_check.c          |  10 +
 tools/testing/radix-tree/test.c               |  13 +-
 tools/testing/radix-tree/test.h               |   7 +-
 17 files changed, 845 insertions(+), 347 deletions(-)
 create mode 100644 tools/testing/radix-tree/generated/autoconf.h
 create mode 100644 tools/testing/radix-tree/linux/init.h
 create mode 100644 tools/testing/radix-tree/multiorder.c

-- 
2.8.0.rc3

WARNING: multiple messages have this Message-ID (diff)
From: Matthew Wilcox <willy@linux.intel.com>
To: linux-kernel@vger.kernel.org, Andrew Morton <akpm@linux-foundation.org>
Cc: Matthew Wilcox <willy@linux.intel.com>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	Konstantin Khlebnikov <koct9i@gmail.com>,
	Kirill Shutemov <kirill.shutemov@linux.intel.com>,
	Jan Kara <jack@suse.com>, Neil Brown <neilb@suse.de>,
	Ross Zwisler <ross.zwisler@linux.intel.com>
Subject: [PATCH v2 00/29] Radix tree multiorder fixes
Date: Thu, 14 Apr 2016 10:16:21 -0400	[thread overview]
Message-ID: <1460643410-30196-1-git-send-email-willy@linux.intel.com> (raw)

I must apologise for commit f96d18ff84 which left the impression that
the support for multiorder radix tree entries was functional.  As soon as
Ross tried to use it, it became apparent that my testing was completely
inadequate, and it didn't even work a little bit for orders that were
not a multiple of shift.

This series of patches is the result of about 6 weeks of redesign,
reimplementation, testing, arguing and hair-pulling.  The great news is
that the test-suite is now far better than it was.  That's reflected in
the diffstat for the test-suite alone:
 12 files changed, 436 insertions(+), 28 deletions(-)

The highlight for users of the tree is that the restriction on the order
of inserted entries being >= RADIX_TREE_MAP_SHIFT is now gone; the radix
tree now supports any order between 0 and 64.

For those who are interested in how the tree works, patch 9 is probably
the most interesting one as it introduces the new machinery for handling
sibling entries.

I've tried to be fair in attributing authorship to the person who
contributed the majority of the code in each patch; Ross has been an
invaluable partner in the development of this support and it's fair to
say that each of us has code in every commit.

I should also express my appreciation of the 0day testing.  It prompted
me that I was bloating the tinyconfig in an unacceptable way, and it
bisected to a commit which contained a rather nasty memory-corruption bug.

Changes since v1:

 - Rename get_sibling_offset() to get_slot_offset()
 - Use get_slot_offset() in radix_tree_insert() instead of doing arithmetic
 - Use get_slot_offset() in radix_tree_descend()
 - Some whitespace fixes
 - Fix race in radix_tree_next_chunk() against a multiorder insertion
 - Reduce the nuber of ifdefs by introducing iter_shift()
 - Fix some minor bugs in radix_tree_dump()
 - Merge the commit to fix shrinking bugs with the commit to add a test case
   for the shrinking code
 - Fix a bug in radix_tree_range_tag_if_tagged() if the first index was an
   alias of a multiorder entry
 - Add more test-cases to the test suite
 - Make __radix_tree_lookup fit the same pattern as the other tree walkers

Matthew Wilcox (18):
  radix-tree: Introduce radix_tree_empty
  radix tree test suite: Fix build
  radix tree test suite: Add tests for radix_tree_locate_item()
  Introduce CONFIG_RADIX_TREE_MULTIORDER
  radix-tree: Add missing sibling entry functionality
  radix-tree: Fix sibling entry insertion
  radix-tree: Fix deleting a multi-order entry through an alias
  radix-tree: Remove restriction on multi-order entries
  radix-tree: Introduce radix_tree_load_root()
  radix-tree: Fix extending the tree for multi-order entries at offset 0
  radix tree test suite: Start adding multiorder tests
  radix-tree: Fix several shrinking bugs with multiorder entries
  radix-tree: Rewrite __radix_tree_lookup
  radix-tree: Fix multiorder BUG_ON in radix_tree_insert
  radix-tree: Fix radix_tree_create for sibling entries
  radix-tree: Rewrite radix_tree_locate_item
  radix-tree: Fix radix_tree_range_tag_if_tagged() for multiorder
    entries
  radix-tree: Add copyright statements

Ross Zwisler (11):
  radix tree test suite: Allow testing other fan-out values
  radix tree test suite: keep regression test runs short
  radix tree test suite: rebuild when headers change
  radix-tree: remove unused looping macros
  radix-tree: add support for multi-order iterating
  radix tree test suite: multi-order iteration test
  radix-tree: Rewrite radix_tree_tag_set
  radix-tree: Rewrite radix_tree_tag_clear
  radix-tree: Rewrite radix_tree_tag_get
  radix-tree test suite: add multi-order tag test
  radix-tree: Fix radix_tree_dump() for multi-order entries

 include/linux/radix-tree.h                    | 106 +++--
 kernel/irq/irqdomain.c                        |   7 +-
 lib/Kconfig                                   |   3 +
 lib/radix-tree.c                              | 611 ++++++++++++++------------
 mm/Kconfig                                    |   1 +
 tools/testing/radix-tree/Makefile             |   4 +-
 tools/testing/radix-tree/generated/autoconf.h |   3 +
 tools/testing/radix-tree/linux/init.h         |   0
 tools/testing/radix-tree/linux/kernel.h       |  15 +-
 tools/testing/radix-tree/linux/slab.h         |   1 -
 tools/testing/radix-tree/linux/types.h        |   7 +-
 tools/testing/radix-tree/main.c               |  71 ++-
 tools/testing/radix-tree/multiorder.c         | 326 ++++++++++++++
 tools/testing/radix-tree/regression2.c        |   7 -
 tools/testing/radix-tree/tag_check.c          |  10 +
 tools/testing/radix-tree/test.c               |  13 +-
 tools/testing/radix-tree/test.h               |   7 +-
 17 files changed, 845 insertions(+), 347 deletions(-)
 create mode 100644 tools/testing/radix-tree/generated/autoconf.h
 create mode 100644 tools/testing/radix-tree/linux/init.h
 create mode 100644 tools/testing/radix-tree/multiorder.c

-- 
2.8.0.rc3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

             reply	other threads:[~2016-04-14 14:22 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-04-14 14:16 Matthew Wilcox [this message]
2016-04-14 14:16 ` [PATCH v2 00/29] Radix tree multiorder fixes Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 01/29] radix-tree: Introduce radix_tree_empty Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-05-03 14:24   ` Jan Kara
2016-05-03 14:24     ` Jan Kara
2016-04-14 14:16 ` [PATCH v2 02/29] radix tree test suite: Fix build Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 03/29] radix tree test suite: Add tests for radix_tree_locate_item() Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 04/29] radix tree test suite: Allow testing other fan-out values Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 05/29] radix tree test suite: keep regression test runs short Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 06/29] radix tree test suite: rebuild when headers change Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 07/29] radix-tree: remove unused looping macros Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 08/29] Introduce CONFIG_RADIX_TREE_MULTIORDER Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 09/29] radix-tree: Add missing sibling entry functionality Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 10/29] radix-tree: Fix sibling entry insertion Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 11/29] radix-tree: Fix deleting a multi-order entry through an alias Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 12/29] radix-tree: Remove restriction on multi-order entries Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 13/29] radix-tree: Introduce radix_tree_load_root() Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 14/29] radix-tree: Fix extending the tree for multi-order entries at offset 0 Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 15/29] radix tree test suite: Start adding multiorder tests Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 16/29] radix-tree: Fix several shrinking bugs with multiorder entries Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 17/29] radix-tree: Rewrite __radix_tree_lookup Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 18/29] radix-tree: Fix multiorder BUG_ON in radix_tree_insert Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 19/29] radix-tree: add support for multi-order iterating Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 20/29] radix tree test suite: multi-order iteration test Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 21/29] radix-tree: Rewrite radix_tree_tag_set Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 22/29] radix-tree: Rewrite radix_tree_tag_clear Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 23/29] radix-tree: Rewrite radix_tree_tag_get Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 24/29] radix-tree test suite: add multi-order tag test Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 25/29] radix-tree: Fix radix_tree_create for sibling entries Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 26/29] radix-tree: Rewrite radix_tree_locate_item Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 27/29] radix-tree: Fix radix_tree_range_tag_if_tagged() for multiorder entries Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 28/29] radix-tree: Fix radix_tree_dump() for multi-order entries Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox
2016-04-14 14:16 ` [PATCH v2 29/29] radix-tree: Add copyright statements Matthew Wilcox
2016-04-14 14:16   ` Matthew Wilcox

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1460643410-30196-1-git-send-email-willy@linux.intel.com \
    --to=willy@linux.intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=jack@suse.com \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=koct9i@gmail.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=neilb@suse.de \
    --cc=ross.zwisler@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.