From: Elijah Newren <newren@gmail.com> To: Jeff King <peff@peff.net> Cc: Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>, Git Mailing List <git@vger.kernel.org> Subject: Re: [PATCH v2 04/10] hashmap: introduce a new hashmap_partial_clear() Date: Fri, 30 Oct 2020 09:03:38 -0700 [thread overview] Message-ID: <CABPp-BEV6ZNsA_qqV67P=W4gASLGoggoR4wP2R6kf0eNfcsDCw@mail.gmail.com> (raw) In-Reply-To: <20201030134131.GD3277724@coredump.intra.peff.net> On Fri, Oct 30, 2020 at 6:41 AM Jeff King <peff@peff.net> wrote: > > On Tue, Oct 13, 2020 at 12:40:44AM +0000, Elijah Newren via GitGitGadget wrote: > > > merge-ort is a heavy user of strmaps, which are built on hashmap.[ch]. > > reset_maps() in merge-ort was taking about 12% of overall runtime in my > > testcase involving rebasing 35 patches of linux.git across a big rename. > > reset_maps() was calling hashmap_free() followed by hashmap_init(), > > meaning that not only was it freeing all the memory associated with each > > of the strmaps just to immediately allocate a new array again, it was > > allocating a new array that wasy likely smaller than needed (thus > > s/wasy/was/ Thanks; will fix. > > resulting in later need to rehash things). The ending size of the map > > table on the previous commit was likely almost perfectly sized for the > > next commit we wanted to pick, and not dropping and reallocating the > > table immediately is a win. > > > > Add some new API to hashmap to clear a hashmap of entries without > > freeing map->table (and instead only zeroing it out like alloc_table() > > would do, along with zeroing the count of items in the table and the > > shrink_at field). > > This seems like a reasonable optimization to make, and doesn't make the > API significantly more complicated. I'd expect the allocation of actual > entry objects to dwarf the table allocation, but I guess: > > - you'll deal with the individual entries later using a mempool > > - it's not just the allocation, but the re-insertion of the entries as > we grow > > It would be nice if we had some actual perf numbers to report here, so > we could know exactly how much it was buying us. But I guess things are > a bit out-of-order there. You want to do this series first and then > build merge-ort on top as a user. We could introduce the basic data > structure first, then merge-ort, and then start applying optimizations > with real-world measurements. But I'm not sure it's worth the amount of > time you'd have to spend to reorganize in that way. Yeah, the perf benefits didn't really come until I added a strmap_clear() based on this, so as you discovered I put perf numbers in patch 7 of this series. Should I add a mention of the later commit message at this point in the series? > > hashmap.c | 39 +++++++++++++++++++++++++++------------ > > hashmap.h | 13 ++++++++++++- > > The implementation itself looks correct to me. I already mentioned my > thoughts on naming in patch 1. I'll circle back to that when I comment on patch 1...
next prev parent reply other threads:[~2020-10-30 16:03 UTC|newest] Thread overview: 144+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-08-21 18:52 [PATCH 0/5] Add struct strmap and associated utility functions Elijah Newren via GitGitGadget 2020-08-21 18:52 ` [PATCH 1/5] hashmap: add usage documentation explaining hashmap_free[_entries]() Elijah Newren via GitGitGadget 2020-08-21 19:22 ` Jeff King 2020-08-21 18:52 ` [PATCH 2/5] strmap: new utility functions Elijah Newren via GitGitGadget 2020-08-21 19:48 ` Jeff King 2020-08-21 18:52 ` [PATCH 3/5] strmap: add more " Elijah Newren via GitGitGadget 2020-08-21 19:58 ` Jeff King 2020-08-21 18:52 ` [PATCH 4/5] strmap: add strdup_strings option Elijah Newren via GitGitGadget 2020-08-21 20:01 ` Jeff King 2020-08-21 20:41 ` Elijah Newren 2020-08-21 21:03 ` Jeff King 2020-08-21 22:25 ` Elijah Newren 2020-08-28 7:08 ` Jeff King 2020-08-28 17:20 ` Elijah Newren 2020-08-21 18:52 ` [PATCH 5/5] strmap: add functions facilitating use as a string->int map Elijah Newren via GitGitGadget 2020-08-21 20:10 ` Jeff King 2020-08-21 20:51 ` Elijah Newren 2020-08-21 21:05 ` Jeff King 2020-08-21 20:16 ` [PATCH 0/5] Add struct strmap and associated utility functions Jeff King 2020-08-21 21:33 ` Elijah Newren 2020-08-21 22:28 ` Elijah Newren 2020-08-28 7:03 ` Jeff King 2020-08-28 15:29 ` Elijah Newren 2020-09-01 9:27 ` Jeff King 2020-10-13 0:40 ` [PATCH v2 00/10] " Elijah Newren via GitGitGadget 2020-10-13 0:40 ` [PATCH v2 01/10] hashmap: add usage documentation explaining hashmap_free[_entries]() Elijah Newren via GitGitGadget 2020-10-30 12:50 ` Jeff King 2020-10-30 19:55 ` Elijah Newren 2020-11-03 16:26 ` Jeff King 2020-11-03 16:48 ` Elijah Newren 2020-10-13 0:40 ` [PATCH v2 02/10] hashmap: adjust spacing to fix argument alignment Elijah Newren via GitGitGadget 2020-10-30 12:51 ` Jeff King 2020-10-13 0:40 ` [PATCH v2 03/10] hashmap: allow re-use after hashmap_free() Elijah Newren via GitGitGadget 2020-10-30 13:35 ` Jeff King 2020-10-30 15:37 ` Elijah Newren 2020-11-03 16:08 ` Jeff King 2020-11-03 16:16 ` Elijah Newren 2020-10-13 0:40 ` [PATCH v2 04/10] hashmap: introduce a new hashmap_partial_clear() Elijah Newren via GitGitGadget 2020-10-30 13:41 ` Jeff King 2020-10-30 16:03 ` Elijah Newren [this message] 2020-11-03 16:10 ` Jeff King 2020-10-13 0:40 ` [PATCH v2 05/10] strmap: new utility functions Elijah Newren via GitGitGadget 2020-10-30 14:12 ` Jeff King 2020-10-30 16:26 ` Elijah Newren 2020-10-13 0:40 ` [PATCH v2 06/10] strmap: add more " Elijah Newren via GitGitGadget 2020-10-30 14:23 ` Jeff King 2020-10-30 16:43 ` Elijah Newren 2020-11-03 16:12 ` Jeff King 2020-10-13 0:40 ` [PATCH v2 07/10] strmap: enable faster clearing and reusing of strmaps Elijah Newren via GitGitGadget 2020-10-30 14:27 ` Jeff King 2020-10-13 0:40 ` [PATCH v2 08/10] strmap: add functions facilitating use as a string->int map Elijah Newren via GitGitGadget 2020-10-30 14:39 ` Jeff King 2020-10-30 17:28 ` Elijah Newren 2020-11-03 16:20 ` Jeff King 2020-11-03 16:46 ` Elijah Newren 2020-10-13 0:40 ` [PATCH v2 09/10] strmap: add a strset sub-type Elijah Newren via GitGitGadget 2020-10-30 14:44 ` Jeff King 2020-10-30 18:02 ` Elijah Newren 2020-10-13 0:40 ` [PATCH v2 10/10] strmap: enable allocations to come from a mem_pool Elijah Newren via GitGitGadget 2020-10-30 14:56 ` Jeff King 2020-10-30 19:31 ` Elijah Newren 2020-11-03 16:24 ` Jeff King 2020-11-02 18:55 ` [PATCH v3 00/13] Add struct strmap and associated utility functions Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 01/13] hashmap: add usage documentation explaining hashmap_free[_entries]() Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 02/13] hashmap: adjust spacing to fix argument alignment Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 03/13] hashmap: allow re-use after hashmap_free() Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 04/13] hashmap: introduce a new hashmap_partial_clear() Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 05/13] hashmap: provide deallocation function names Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 06/13] strmap: new utility functions Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 07/13] strmap: add more " Elijah Newren via GitGitGadget 2020-11-04 20:13 ` Jeff King 2020-11-04 20:24 ` Elijah Newren 2020-11-02 18:55 ` [PATCH v3 08/13] strmap: enable faster clearing and reusing of strmaps Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 09/13] strmap: add functions facilitating use as a string->int map Elijah Newren via GitGitGadget 2020-11-04 20:21 ` Jeff King 2020-11-02 18:55 ` [PATCH v3 10/13] strmap: add a strset sub-type Elijah Newren via GitGitGadget 2020-11-04 20:31 ` Jeff King 2020-11-02 18:55 ` [PATCH v3 11/13] strmap: enable allocations to come from a mem_pool Elijah Newren via GitGitGadget 2020-11-02 18:55 ` [PATCH v3 12/13] strmap: take advantage of FLEXPTR_ALLOC_STR when relevant Elijah Newren via GitGitGadget 2020-11-04 20:43 ` Jeff King 2020-11-02 18:55 ` [PATCH v3 13/13] Use new HASHMAP_INIT macro to simplify hashmap initialization Elijah Newren via GitGitGadget 2020-11-04 20:48 ` Jeff King 2020-11-04 20:52 ` [PATCH v3 00/13] Add struct strmap and associated utility functions Jeff King 2020-11-04 22:20 ` Elijah Newren 2020-11-05 0:22 ` [PATCH v4 " Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 01/13] hashmap: add usage documentation explaining hashmap_free[_entries]() Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 02/13] hashmap: adjust spacing to fix argument alignment Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 03/13] hashmap: allow re-use after hashmap_free() Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 04/13] hashmap: introduce a new hashmap_partial_clear() Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 05/13] hashmap: provide deallocation function names Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 06/13] strmap: new utility functions Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 07/13] strmap: add more " Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 08/13] strmap: enable faster clearing and reusing of strmaps Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 09/13] strmap: add functions facilitating use as a string->int map Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 10/13] strmap: add a strset sub-type Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 11/13] strmap: enable allocations to come from a mem_pool Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 12/13] strmap: take advantage of FLEXPTR_ALLOC_STR when relevant Elijah Newren via GitGitGadget 2020-11-05 0:22 ` [PATCH v4 13/13] Use new HASHMAP_INIT macro to simplify hashmap initialization Elijah Newren via GitGitGadget 2020-11-05 13:29 ` [PATCH v4 00/13] Add struct strmap and associated utility functions Jeff King 2020-11-05 20:25 ` Junio C Hamano 2020-11-05 21:17 ` Jeff King 2020-11-05 21:22 ` Elijah Newren 2020-11-05 22:15 ` Junio C Hamano 2020-11-06 0:24 ` [PATCH v5 00/15] " Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 01/15] hashmap: add usage documentation explaining hashmap_free[_entries]() Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 02/15] hashmap: adjust spacing to fix argument alignment Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 03/15] hashmap: allow re-use after hashmap_free() Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 04/15] hashmap: introduce a new hashmap_partial_clear() Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 05/15] hashmap: provide deallocation function names Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 06/15] strmap: new utility functions Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 07/15] strmap: add more " Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 08/15] strmap: enable faster clearing and reusing of strmaps Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 09/15] strmap: add functions facilitating use as a string->int map Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 10/15] strmap: split create_entry() out of strmap_put() Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 11/15] strmap: add a strset sub-type Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 12/15] strmap: enable allocations to come from a mem_pool Elijah Newren via GitGitGadget 2020-11-11 17:33 ` Phillip Wood 2020-11-11 18:49 ` Elijah Newren 2020-11-11 19:01 ` Jeff King 2020-11-11 20:34 ` Chris Torek 2020-11-06 0:24 ` [PATCH v5 13/15] strmap: take advantage of FLEXPTR_ALLOC_STR when relevant Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 14/15] Use new HASHMAP_INIT macro to simplify hashmap initialization Elijah Newren via GitGitGadget 2020-11-06 0:24 ` [PATCH v5 15/15] shortlog: use strset from strmap.h Elijah Newren via GitGitGadget 2020-11-06 2:00 ` [PATCH v5 00/15] Add struct strmap and associated utility functions Junio C Hamano 2020-11-06 2:42 ` Elijah Newren 2020-11-06 2:48 ` Jeff King 2020-11-06 17:32 ` Junio C Hamano 2020-11-11 20:02 ` [PATCH v6 " Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 01/15] hashmap: add usage documentation explaining hashmap_free[_entries]() Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 02/15] hashmap: adjust spacing to fix argument alignment Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 03/15] hashmap: allow re-use after hashmap_free() Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 04/15] hashmap: introduce a new hashmap_partial_clear() Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 05/15] hashmap: provide deallocation function names Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 06/15] strmap: new utility functions Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 07/15] strmap: add more " Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 08/15] strmap: enable faster clearing and reusing of strmaps Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 09/15] strmap: add functions facilitating use as a string->int map Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 10/15] strmap: split create_entry() out of strmap_put() Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 11/15] strmap: add a strset sub-type Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 12/15] strmap: enable allocations to come from a mem_pool Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 13/15] strmap: take advantage of FLEXPTR_ALLOC_STR when relevant Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 14/15] Use new HASHMAP_INIT macro to simplify hashmap initialization Elijah Newren via GitGitGadget 2020-11-11 20:02 ` [PATCH v6 15/15] shortlog: use strset from strmap.h Elijah Newren via GitGitGadget 2020-11-11 20:07 ` [PATCH v6 00/15] Add struct strmap and associated utility functions Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CABPp-BEV6ZNsA_qqV67P=W4gASLGoggoR4wP2R6kf0eNfcsDCw@mail.gmail.com' \ --to=newren@gmail.com \ --cc=git@vger.kernel.org \ --cc=gitgitgadget@gmail.com \ --cc=peff@peff.net \ --subject='Re: [PATCH v2 04/10] hashmap: introduce a new hashmap_partial_clear()' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).