From: Jeff King <peff@peff.net> To: git@vger.kernel.org Subject: [PATCH 4/6] t/perf: add perf tests for fetches from a bitmapped server Date: Tue, 21 Aug 2018 15:06:58 -0400 [thread overview] Message-ID: <20180821190658.GD30764@sigill.intra.peff.net> (raw) In-Reply-To: <20180821184140.GA24165@sigill.intra.peff.net> A server with bitmapped packs can serve a clone very quickly. However, fetches are not necessarily made any faster, because we spend a lot less time in object traversal (which is what bitmaps help with) and more time finding deltas (because we may have to throw out on-disk deltas if the client does not have the base). As a first step to making this faster, this patch introduces a new perf script to measure fetches into a repo of various ages from a fully-bitmapped server. We separately measure the work done by the server (in pack-objects) and that done by the client (in index-pack). Furthermore, we measure the size of the resulting pack. Breaking it down like this (instead of just doing a regular "git fetch") lets us see how much each side benefits from any changes. And since we know the pack size, if we estimate the network speed, then one could calculate a complete wall-clock time for the operation (though the script does not do this automatically). Signed-off-by: Jeff King <peff@peff.net> --- t/perf/p5311-pack-bitmaps-fetch.sh | 45 ++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100755 t/perf/p5311-pack-bitmaps-fetch.sh diff --git a/t/perf/p5311-pack-bitmaps-fetch.sh b/t/perf/p5311-pack-bitmaps-fetch.sh new file mode 100755 index 0000000000..b04575951f --- /dev/null +++ b/t/perf/p5311-pack-bitmaps-fetch.sh @@ -0,0 +1,45 @@ +#!/bin/sh + +test_description='performance of fetches from bitmapped packs' +. ./perf-lib.sh + +test_perf_default_repo + +test_expect_success 'create bitmapped server repo' ' + git config pack.writebitmaps true && + git config pack.writebitmaphashcache true && + git repack -ad +' + +# simulate a fetch from a repository that last fetched N days ago, for +# various values of N. We do so by following the first-parent chain, +# and assume the first entry in the chain that is N days older than the current +# HEAD is where the HEAD would have been then. +for days in 1 2 4 8 16 32 64 128; do + title=$(printf '%10s' "($days days)") + test_expect_success "setup revs from $days days ago" ' + now=$(git log -1 --format=%ct HEAD) && + then=$(($now - ($days * 86400))) && + tip=$(git rev-list -1 --first-parent --until=$then HEAD) && + { + echo HEAD && + echo ^$tip + } >revs + ' + + test_perf "server $title" ' + git pack-objects --stdout --revs \ + --thin --delta-base-offset \ + <revs >tmp.pack + ' + + test_size "size $title" ' + wc -c <tmp.pack + ' + + test_perf "client $title" ' + git index-pack --stdin --fix-thin <tmp.pack + ' +done + +test_done -- 2.19.0.rc0.398.g138a08f6f6
next prev parent reply other threads:[~2018-08-21 19:07 UTC|newest] Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-08-21 18:41 [PATCH] test-tool.h: include git-compat-util.h Jeff King 2018-08-21 19:03 ` Junio C Hamano 2018-08-21 19:06 ` [PATCH 1/6] t/perf: factor boilerplate out of test_perf Jeff King 2018-08-21 19:06 ` [PATCH 2/6] t/perf: factor out percent calculations Jeff King 2018-08-21 19:06 ` [PATCH 3/6] t/perf: add infrastructure for measuring sizes Jeff King 2018-08-22 13:40 ` Derrick Stolee 2018-08-22 15:31 ` Jeff King 2018-08-21 19:06 ` Jeff King [this message] 2018-08-21 19:07 ` [PATCH 5/6] pack-bitmap: save "have" bitmap from walk Jeff King 2018-08-21 19:47 ` Derrick Stolee 2018-08-21 19:54 ` Jeff King 2018-08-31 15:23 ` Ævar Arnfjörð Bjarmason 2018-08-31 22:55 ` Jeff King 2018-09-01 7:41 ` [PATCH 0/4] un-breaking pack-objects with bitmaps Jeff King 2018-09-01 7:44 ` [PATCH 1/4] bitmap_has_sha1_in_uninteresting(): drop BUG check Jeff King 2018-09-01 7:48 ` [PATCH 2/4] t5310: test delta reuse with bitmaps Jeff King 2018-09-01 8:03 ` Jeff King 2018-09-01 20:29 ` Ævar Arnfjörð Bjarmason 2018-09-01 22:46 ` Ben Peart 2018-09-02 5:51 ` Jeff King 2018-09-04 19:05 ` Stefan Beller 2018-09-04 19:45 ` Junio C Hamano 2018-09-04 20:02 ` Jeff King 2018-09-01 7:49 ` [PATCH 3/4] traverse_bitmap_commit_list(): don't free result Jeff King 2018-09-01 7:50 ` [PATCH 4/4] pack-bitmap: drop "loaded" flag Jeff King 2018-09-04 19:30 ` [PATCH 0/4] un-breaking pack-objects with bitmaps Stefan Beller 2018-09-04 20:03 ` Jeff King 2018-09-08 6:43 ` Ævar Arnfjörð Bjarmason 2018-09-10 16:53 ` Junio C Hamano 2018-09-10 18:48 ` Jeff King 2018-09-10 19:23 ` Junio C Hamano 2018-08-21 19:07 ` [PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects Jeff King 2018-08-21 19:43 ` Junio C Hamano 2018-08-21 19:50 ` Junio C Hamano 2018-08-21 20:07 ` Jeff King 2018-08-21 20:14 ` Jeff King 2018-08-21 20:52 ` Junio C Hamano 2018-08-21 21:30 ` Jeff King 2018-08-21 20:57 ` Junio C Hamano 2018-08-21 21:32 ` Jeff King 2018-08-23 0:43 ` [PATCH 0/9] trailer-parsing false positives Jeff King 2018-08-23 0:44 ` [PATCH 1/9] trailer: use size_t for string offsets Jeff King 2018-08-23 0:45 ` [PATCH 2/9] trailer: use size_t for iterating trailer list Jeff King 2018-08-23 0:46 ` [PATCH 3/9] trailer: pass process_trailer_opts to trailer_info_get() Jeff King 2018-08-23 0:48 ` [PATCH 4/9] interpret-trailers: tighten check for "---" patch boundary Jeff King 2018-08-23 0:49 ` [PATCH 5/9] interpret-trailers: allow suppressing "---" divider Jeff King 2018-08-23 0:50 ` [PATCH 6/9] pretty, ref-filter: format %(trailers) with no_divider option Jeff King 2018-08-23 0:50 ` [PATCH 7/9] sequencer: ignore "---" divider when parsing trailers Jeff King 2018-08-23 0:50 ` [PATCH 8/9] append_signoff: use size_t for string offsets Jeff King 2018-08-23 0:51 ` [PATCH 9/9] sequencer: handle ignore_footer when parsing trailers Jeff King 2018-08-23 18:30 ` [PATCH 0/9] trailer-parsing false positives Junio C Hamano 2018-08-24 7:26 ` Jeff King 2018-08-21 20:00 ` [PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects Jeff King -- strict thread matches above, loose matches on Subject: below -- 2018-08-17 20:54 [PATCH 0/6] reuse on-disk deltas for fetches with bitmaps Jeff King 2018-08-17 20:57 ` [PATCH 4/6] t/perf: add perf tests for fetches from a bitmapped server Jeff King 2014-03-26 7:22 [PATCH/RFC 0/6] reuse deltas found by bitmaps Jeff King 2014-03-26 7:22 ` [PATCH 4/6] t/perf: add perf tests for fetches from a bitmapped server Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180821190658.GD30764@sigill.intra.peff.net \ --to=peff@peff.net \ --cc=git@vger.kernel.org \ --subject='Re: [PATCH 4/6] t/perf: add perf tests for fetches from a bitmapped server' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).