From: Junio C Hamano <email@example.com> To: Eric Wong <firstname.lastname@example.org> Cc: Colin Stolley <email@example.com>, firstname.lastname@example.org Subject: Re: hashmap vs khash? Re: [PATCH] packfile.c: speed up loading lots of packfiles. Date: Sat, 30 Nov 2019 09:36:47 -0800 Message-ID: <email@example.com> (raw) In-Reply-To: <20191128004202.GA25910@dcvr> (Eric Wong's message of "Thu, 28 Nov 2019 00:42:02 +0000") Eric Wong <firstname.lastname@example.org> writes: > Colin Stolley <email@example.com> wrote: >> When loading packfiles on start-up, we traverse the internal packfile >> list once per file to avoid reloading packfiles that have already >> been loaded. This check runs in quadratic time, so for poorly >> maintained repos with a large number of packfiles, it can be pretty >> slow. > > Cool! Thanks for looking into this, and I've been having > trouble in that department with big alternates files. > >> Add a hashmap containing the packfile names as we load them so that >> the average runtime cost of checking for already-loaded packs becomes >> constant. > > Btw, would you have time to do a comparison against khash? > > AFAIK hashmap predates khash in git; and hashmap was optimized > for removal. Removals don't seem to be a problem for pack > loading. > > I'm interested in exploring the removing of hashmap entirely in > favor of khash to keep our codebase smaller and easier-to-learn. > khash shows up more in other projects, and ought to have better > cache-locality. Sounds fun ;-) and useful.
next prev parent reply index Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-27 22:24 Colin Stolley 2019-11-28 0:42 ` hashmap vs khash? " Eric Wong 2019-11-30 17:36 ` Junio C Hamano [this message] 2019-12-02 14:39 ` Jeff King 2019-12-02 17:40 ` SZEDER Gábor 2019-12-02 19:42 ` Jeff King 2019-12-03 6:17 ` Taylor Blau 2019-12-03 15:34 ` Jeff King 2019-12-03 16:04 ` Junio C Hamano 2019-12-03 17:33 ` Colin Stolley 2019-12-03 22:18 ` Jeff King 2019-12-04 18:15 ` Junio C Hamano 2019-12-03 22:17 ` Jeff King 2019-12-04 4:23 ` Jonathan Nieder 2019-12-03 6:19 ` Taylor Blau
Reply instructions: You may reply publically to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Git Mailing List Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/git/0 git/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 git git/ https://lore.kernel.org/git \ email@example.com public-inbox-index git Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git