From: Eric Wong <email@example.com> To: Colin Stolley <firstname.lastname@example.org> Cc: email@example.com Subject: hashmap vs khash? Re: [PATCH] packfile.c: speed up loading lots of packfiles. Date: Thu, 28 Nov 2019 00:42:02 +0000 Message-ID: <20191128004202.GA25910@dcvr> (raw) In-Reply-To: <20191127222453.GA3765@owl.colinstolley.com> Colin Stolley <firstname.lastname@example.org> wrote: > When loading packfiles on start-up, we traverse the internal packfile > list once per file to avoid reloading packfiles that have already > been loaded. This check runs in quadratic time, so for poorly > maintained repos with a large number of packfiles, it can be pretty > slow. Cool! Thanks for looking into this, and I've been having trouble in that department with big alternates files. > Add a hashmap containing the packfile names as we load them so that > the average runtime cost of checking for already-loaded packs becomes > constant. Btw, would you have time to do a comparison against khash? AFAIK hashmap predates khash in git; and hashmap was optimized for removal. Removals don't seem to be a problem for pack loading. I'm interested in exploring the removing of hashmap entirely in favor of khash to keep our codebase smaller and easier-to-learn. khash shows up more in other projects, and ought to have better cache-locality. Thanks again.
next prev parent reply index Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-27 22:24 Colin Stolley 2019-11-28 0:42 ` Eric Wong [this message] 2019-11-30 17:36 ` hashmap vs khash? " Junio C Hamano 2019-12-02 14:39 ` Jeff King 2019-12-02 17:40 ` SZEDER Gábor 2019-12-02 19:42 ` Jeff King 2019-12-03 6:17 ` Taylor Blau 2019-12-03 15:34 ` Jeff King 2019-12-03 16:04 ` Junio C Hamano 2019-12-03 17:33 ` Colin Stolley 2019-12-03 22:18 ` Jeff King 2019-12-04 18:15 ` Junio C Hamano 2019-12-03 22:17 ` Jeff King 2019-12-04 4:23 ` Jonathan Nieder 2019-12-03 6:19 ` Taylor Blau
Reply instructions: You may reply publically to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191128004202.GA25910@dcvr \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Git Mailing List Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/git/0 git/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 git git/ https://lore.kernel.org/git \ firstname.lastname@example.org public-inbox-index git Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git