From: Taylor Blau <me@ttaylorr.com>
To: Derrick Stolee <derrickstolee@github.com>
Cc: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
rsbecker@nexbridge.com, "'Jonathan Nieder'" <jrnieder@gmail.com>,
git@vger.kernel.org, gitster@pobox.com, larsxschneider@gmail.com,
tytso@mit.edu
Subject: Re: adding new 32-bit on-disk (unsigned) timestamp formats (was: [PATCH v5 02/17] pack-mtimes: support reading .mtimes files)
Date: Wed, 25 May 2022 17:13:31 -0400 [thread overview]
Message-ID: <Yo6b+8sixGAqMm/x@nand.local> (raw)
In-Reply-To: <32db3720-e9c8-e192-6278-c55855ce1d3e@github.com>
On Wed, May 25, 2022 at 09:30:55AM -0400, Derrick Stolee wrote:
> On 5/25/2022 5:11 AM, Ævar Arnfjörð Bjarmason wrote:
> > I must say that I really don't like this part of the format. Is it
> > really necessary to optimize the storage space here in a way that leaves
> > open questions about future time_t compatibility, and having to
> > introduce the first use of unsigned 32 bit timestamps to git's codebase?
>
> The commit-graph file format uses unsigned 34-bit timestamps (packed
> with 30-bit topological levels in the CDAT chunk), so this "not-64-bit
> signed timestamps" thing is something we've done before.
>
> > Yes, this is its own self-contained format, so we don't *need* time_t
> > here, but it's also really handy if we can eventually consistently use
> > 64 time_t everywhere and not worry about any compatibility issues, or
> > unsigned v.s. signed, or to create our own little ext4-like signed 32
> > bit timestamp format.
>
> We can also use a new file format version when it is necessary. We
> have a lot of time to add that detail without overly complicating the
> format right now.
>
> > If we really are trying to micro-optimize storage space here I'm willing
> > to bet that this is still a bad/premature optimization. There's much
> > better ways to store this sort of data in a compact way if that's the
> > concern. E.g. you'd store a 64 bit "base" timestamp in the header for
> > the first entry, and have smaller (signed) "delta" timestamps storing
> > offsets from that "base" timestamp.
>
> This is a good idea for a v2 format when that is necessary.
I agree here.
I'm not opposed to such a change (or even being the one to work on it!),
but I would encourage us to pursue that change outside of this series,
since it can easily be done on top.
Of course, if we ever did decide to implement 64-bit mtimes, we would
have to maintain support for reading both the 32-bit and 64-bit values.
But I think the code is well-equipped to do that, and it could be done
on top without significant additional complexity.
Thanks,
Taylor
next prev parent reply other threads:[~2022-05-25 21:13 UTC|newest]
Thread overview: 201+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-29 22:25 [PATCH 00/17] cruft packs Taylor Blau
2021-11-29 22:25 ` [PATCH 01/17] Documentation/technical: add cruft-packs.txt Taylor Blau
2021-12-02 14:33 ` Derrick Stolee
2021-12-03 21:53 ` Taylor Blau
2021-12-04 22:20 ` Elijah Newren
2021-12-04 23:32 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 02/17] pack-mtimes: support reading .mtimes files Taylor Blau
2021-12-02 15:06 ` Derrick Stolee
2021-12-02 22:32 ` brian m. carlson
2021-12-03 22:24 ` Taylor Blau
2022-01-07 19:41 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 03/17] pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles' Taylor Blau
2021-11-29 22:25 ` [PATCH 04/17] chunk-format.h: extract oid_version() Taylor Blau
2021-12-02 15:22 ` Derrick Stolee
2021-12-03 22:40 ` Taylor Blau
2021-12-06 17:33 ` Derrick Stolee
2021-11-29 22:25 ` [PATCH 05/17] pack-mtimes: support writing pack .mtimes files Taylor Blau
2021-12-02 15:36 ` Derrick Stolee
2021-12-03 23:04 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 06/17] t/helper: add 'pack-mtimes' test-tool Taylor Blau
2021-12-06 21:16 ` Derrick Stolee
2022-02-23 22:24 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 07/17] builtin/pack-objects.c: return from create_object_entry() Taylor Blau
2021-11-29 22:25 ` [PATCH 08/17] builtin/pack-objects.c: --cruft without expiration Taylor Blau
2021-12-06 21:44 ` Derrick Stolee
2022-03-01 2:48 ` Taylor Blau
2021-12-07 15:17 ` Derrick Stolee
2022-02-23 23:34 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 09/17] reachable: add options to add_unseen_recent_objects_to_traversal Taylor Blau
2021-11-29 22:25 ` [PATCH 10/17] reachable: report precise timestamps from objects in cruft packs Taylor Blau
2021-11-29 22:25 ` [PATCH 11/17] builtin/pack-objects.c: --cruft with expiration Taylor Blau
2021-12-07 15:30 ` Derrick Stolee
2022-02-23 23:35 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 12/17] builtin/repack.c: support generating a cruft pack Taylor Blau
2021-12-05 20:46 ` Junio C Hamano
2022-03-01 2:00 ` Taylor Blau
2021-12-07 15:38 ` Derrick Stolee
2022-02-23 23:37 ` Taylor Blau
2021-11-29 22:25 ` [PATCH 13/17] builtin/repack.c: allow configuring cruft pack generation Taylor Blau
2021-11-29 22:25 ` [PATCH 14/17] builtin/repack.c: use named flags for existing_packs Taylor Blau
2021-11-29 22:25 ` [PATCH 15/17] builtin/repack.c: add cruft packs to MIDX during geometric repack Taylor Blau
2021-11-29 22:25 ` [PATCH 16/17] builtin/gc.c: conditionally avoid pruning objects via loose Taylor Blau
2021-11-29 22:25 ` [PATCH 17/17] sha1-file.c: don't freshen cruft packs Taylor Blau
2021-12-03 19:51 ` [PATCH 00/17] " Junio C Hamano
2021-12-03 20:08 ` Taylor Blau
2021-12-03 20:47 ` Taylor Blau
2022-03-02 0:57 ` [PATCH v2 " Taylor Blau
2022-03-02 0:58 ` [PATCH v2 01/17] Documentation/technical: add cruft-packs.txt Taylor Blau
2022-03-02 0:58 ` [PATCH v2 02/17] pack-mtimes: support reading .mtimes files Taylor Blau
2022-03-02 20:22 ` Derrick Stolee
2022-03-02 21:33 ` Taylor Blau
2022-03-02 0:58 ` [PATCH v2 03/17] pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles' Taylor Blau
2022-03-02 0:58 ` [PATCH v2 04/17] chunk-format.h: extract oid_version() Taylor Blau
2022-03-02 0:58 ` [PATCH v2 05/17] pack-mtimes: support writing pack .mtimes files Taylor Blau
2022-03-02 0:58 ` [PATCH v2 06/17] t/helper: add 'pack-mtimes' test-tool Taylor Blau
2022-03-02 0:58 ` [PATCH v2 07/17] builtin/pack-objects.c: return from create_object_entry() Taylor Blau
2022-03-02 0:58 ` [PATCH v2 08/17] builtin/pack-objects.c: --cruft without expiration Taylor Blau
2022-03-02 0:58 ` [PATCH v2 09/17] reachable: add options to add_unseen_recent_objects_to_traversal Taylor Blau
2022-03-02 20:19 ` Derrick Stolee
2022-03-02 21:28 ` Taylor Blau
2022-03-02 0:58 ` [PATCH v2 10/17] reachable: report precise timestamps from objects in cruft packs Taylor Blau
2022-03-02 0:58 ` [PATCH v2 11/17] builtin/pack-objects.c: --cruft with expiration Taylor Blau
2022-03-02 7:42 ` Junio C Hamano
2022-03-02 15:54 ` Taylor Blau
2022-03-02 19:57 ` Derrick Stolee
2022-03-02 0:58 ` [PATCH v2 12/17] builtin/repack.c: support generating a cruft pack Taylor Blau
2022-03-02 0:58 ` [PATCH v2 13/17] builtin/repack.c: allow configuring cruft pack generation Taylor Blau
2022-03-02 0:58 ` [PATCH v2 14/17] builtin/repack.c: use named flags for existing_packs Taylor Blau
2022-03-02 0:58 ` [PATCH v2 15/17] builtin/repack.c: add cruft packs to MIDX during geometric repack Taylor Blau
2022-03-02 0:58 ` [PATCH v2 16/17] builtin/gc.c: conditionally avoid pruning objects via loose Taylor Blau
2022-03-02 0:58 ` [PATCH v2 17/17] sha1-file.c: don't freshen cruft packs Taylor Blau
2022-03-02 20:23 ` [PATCH v2 00/17] " Derrick Stolee
2022-03-02 21:36 ` Taylor Blau
2022-03-03 0:20 ` [PATCH v3 " Taylor Blau
2022-03-03 0:20 ` [PATCH v3 01/17] Documentation/technical: add cruft-packs.txt Taylor Blau
2022-03-07 18:03 ` Jonathan Nieder
2022-03-22 1:16 ` Taylor Blau
2022-03-22 21:45 ` Jonathan Nieder
2022-03-22 22:02 ` Taylor Blau
2022-03-22 23:04 ` Jonathan Nieder
2022-03-23 1:01 ` Taylor Blau
2022-03-28 18:46 ` Taylor Blau
2022-03-28 20:55 ` Junio C Hamano
2022-03-28 21:21 ` Taylor Blau
2022-03-29 15:59 ` Junio C Hamano
2022-03-30 2:23 ` Taylor Blau
2022-03-30 13:37 ` Junio C Hamano
2022-03-30 17:30 ` Taylor Blau
2022-03-03 0:20 ` [PATCH v3 02/17] pack-mtimes: support reading .mtimes files Taylor Blau
2022-03-03 0:20 ` [PATCH v3 03/17] pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles' Taylor Blau
2022-03-03 0:20 ` [PATCH v3 04/17] chunk-format.h: extract oid_version() Taylor Blau
2022-03-03 16:30 ` Ævar Arnfjörð Bjarmason
2022-03-03 23:32 ` Taylor Blau
2022-03-04 0:16 ` Junio C Hamano
2022-03-03 0:20 ` [PATCH v3 05/17] pack-mtimes: support writing pack .mtimes files Taylor Blau
2022-03-03 16:45 ` Ævar Arnfjörð Bjarmason
2022-03-03 23:35 ` Taylor Blau
2022-03-04 10:40 ` Ævar Arnfjörð Bjarmason
2022-03-03 0:20 ` [PATCH v3 06/17] t/helper: add 'pack-mtimes' test-tool Taylor Blau
2022-03-03 0:21 ` [PATCH v3 07/17] builtin/pack-objects.c: return from create_object_entry() Taylor Blau
2022-03-03 0:21 ` [PATCH v3 08/17] builtin/pack-objects.c: --cruft without expiration Taylor Blau
2022-03-03 0:21 ` [PATCH v3 09/17] reachable: add options to add_unseen_recent_objects_to_traversal Taylor Blau
2022-03-03 0:21 ` [PATCH v3 10/17] reachable: report precise timestamps from objects in cruft packs Taylor Blau
2022-03-03 0:21 ` [PATCH v3 11/17] builtin/pack-objects.c: --cruft with expiration Taylor Blau
2022-03-03 0:21 ` [PATCH v3 12/17] builtin/repack.c: support generating a cruft pack Taylor Blau
2022-03-03 0:21 ` [PATCH v3 13/17] builtin/repack.c: allow configuring cruft pack generation Taylor Blau
2022-03-03 0:21 ` [PATCH v3 14/17] builtin/repack.c: use named flags for existing_packs Taylor Blau
2022-03-03 0:21 ` [PATCH v3 15/17] builtin/repack.c: add cruft packs to MIDX during geometric repack Taylor Blau
2022-03-03 0:21 ` [PATCH v3 16/17] builtin/gc.c: conditionally avoid pruning objects via loose Taylor Blau
2022-03-03 0:21 ` [PATCH v3 17/17] sha1-file.c: don't freshen cruft packs Taylor Blau
2022-03-03 1:29 ` [PATCH v3 00/17] " Derrick Stolee
2022-05-18 23:10 ` [PATCH v4 " Taylor Blau
2022-05-18 23:10 ` [PATCH v4 01/17] Documentation/technical: add cruft-packs.txt Taylor Blau
2022-05-19 14:04 ` Junio C Hamano
2022-05-18 23:10 ` [PATCH v4 02/17] pack-mtimes: support reading .mtimes files Taylor Blau
2022-05-19 10:40 ` Ævar Arnfjörð Bjarmason
2022-05-19 15:21 ` Junio C Hamano
2022-05-20 7:32 ` Ævar Arnfjörð Bjarmason
2022-05-20 22:37 ` Taylor Blau
2022-05-18 23:10 ` [PATCH v4 03/17] pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles' Taylor Blau
2022-05-18 23:11 ` [PATCH v4 04/17] chunk-format.h: extract oid_version() Taylor Blau
2022-05-19 11:44 ` Ævar Arnfjörð Bjarmason
2022-05-18 23:11 ` [PATCH v4 05/17] pack-mtimes: support writing pack .mtimes files Taylor Blau
2022-05-18 23:11 ` [PATCH v4 06/17] t/helper: add 'pack-mtimes' test-tool Taylor Blau
2022-05-18 23:11 ` [PATCH v4 07/17] builtin/pack-objects.c: return from create_object_entry() Taylor Blau
2022-05-18 23:11 ` [PATCH v4 08/17] builtin/pack-objects.c: --cruft without expiration Taylor Blau
2022-05-19 10:04 ` Junio C Hamano
2022-05-19 15:16 ` Junio C Hamano
2022-05-20 22:52 ` Taylor Blau
2022-05-18 23:11 ` [PATCH v4 09/17] reachable: add options to add_unseen_recent_objects_to_traversal Taylor Blau
2022-05-18 23:11 ` [PATCH v4 10/17] reachable: report precise timestamps from objects in cruft packs Taylor Blau
2022-05-18 23:11 ` [PATCH v4 11/17] builtin/pack-objects.c: --cruft with expiration Taylor Blau
2022-05-18 23:11 ` [PATCH v4 12/17] builtin/repack.c: support generating a cruft pack Taylor Blau
2022-05-19 11:29 ` Ævar Arnfjörð Bjarmason
2022-05-20 22:39 ` Taylor Blau
2022-05-18 23:11 ` [PATCH v4 13/17] builtin/repack.c: allow configuring cruft pack generation Taylor Blau
2022-05-18 23:11 ` [PATCH v4 14/17] builtin/repack.c: use named flags for existing_packs Taylor Blau
2022-05-18 23:11 ` [PATCH v4 15/17] builtin/repack.c: add cruft packs to MIDX during geometric repack Taylor Blau
2022-05-19 11:32 ` Ævar Arnfjörð Bjarmason
2022-05-20 22:42 ` Taylor Blau
2022-05-18 23:11 ` [PATCH v4 16/17] builtin/gc.c: conditionally avoid pruning objects via loose Taylor Blau
2022-05-18 23:11 ` [PATCH v4 17/17] sha1-file.c: don't freshen cruft packs Taylor Blau
2022-05-18 23:48 ` [PATCH v4 00/17] " Derrick Stolee
2022-05-20 23:19 ` Junio C Hamano
2022-05-20 23:30 ` Taylor Blau
2022-05-19 11:42 ` [RFC PATCH 0/2] Utility functions for duplicated pack(write) code Ævar Arnfjörð Bjarmason
2022-05-19 11:42 ` [RFC PATCH 1/2] packfile API: add and use a pack_name_to_ext() utility function Ævar Arnfjörð Bjarmason
2022-05-19 15:40 ` Junio C Hamano
2022-05-19 11:42 ` [RFC PATCH 2/2] hash API: add and use a hash_short_id_by_algo() function Ævar Arnfjörð Bjarmason
2022-05-19 15:50 ` Junio C Hamano
2022-05-19 19:07 ` Ævar Arnfjörð Bjarmason
2022-05-19 15:31 ` [RFC PATCH 0/2] Utility functions for duplicated pack(write) code Junio C Hamano
2022-05-19 11:54 ` [PATCH v4 00/17] cruft packs Ævar Arnfjörð Bjarmason
2022-05-20 23:17 ` [PATCH v5 " Taylor Blau
2022-05-20 23:17 ` [PATCH v5 01/17] Documentation/technical: add cruft-packs.txt Taylor Blau
2022-05-20 23:17 ` [PATCH v5 02/17] pack-mtimes: support reading .mtimes files Taylor Blau
2022-05-24 19:32 ` Jonathan Nieder
2022-05-24 19:44 ` rsbecker
2022-05-24 22:25 ` Taylor Blau
2022-05-24 23:24 ` rsbecker
2022-05-25 0:07 ` Taylor Blau
2022-05-25 0:20 ` rsbecker
2022-05-25 9:11 ` adding new 32-bit on-disk (unsigned) timestamp formats (was: [PATCH v5 02/17] pack-mtimes: support reading .mtimes files) Ævar Arnfjörð Bjarmason
2022-05-25 13:30 ` Derrick Stolee
2022-05-25 21:13 ` Taylor Blau [this message]
2022-05-26 0:02 ` Ævar Arnfjörð Bjarmason
2022-05-26 0:12 ` Taylor Blau
2022-05-24 22:21 ` [PATCH v5 02/17] pack-mtimes: support reading .mtimes files Taylor Blau
2022-05-25 7:48 ` Jonathan Nieder
2022-05-25 21:36 ` Taylor Blau
2022-05-25 21:58 ` rsbecker
2022-05-25 22:59 ` Taylor Blau
2022-05-25 23:02 ` Taylor Blau
2022-05-26 0:30 ` Junio C Hamano
2023-06-01 13:01 ` Andreas Schwab
2022-05-20 23:17 ` [PATCH v5 03/17] pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles' Taylor Blau
2022-05-20 23:17 ` [PATCH v5 04/17] chunk-format.h: extract oid_version() Taylor Blau
2022-05-20 23:17 ` [PATCH v5 05/17] pack-mtimes: support writing pack .mtimes files Taylor Blau
2022-05-20 23:17 ` [PATCH v5 06/17] t/helper: add 'pack-mtimes' test-tool Taylor Blau
2022-05-20 23:17 ` [PATCH v5 07/17] builtin/pack-objects.c: return from create_object_entry() Taylor Blau
2022-05-20 23:17 ` [PATCH v5 08/17] builtin/pack-objects.c: --cruft without expiration Taylor Blau
2022-05-20 23:17 ` [PATCH v5 09/17] reachable: add options to add_unseen_recent_objects_to_traversal Taylor Blau
2022-05-20 23:17 ` [PATCH v5 10/17] reachable: report precise timestamps from objects in cruft packs Taylor Blau
2022-05-20 23:18 ` [PATCH v5 11/17] builtin/pack-objects.c: --cruft with expiration Taylor Blau
2022-05-20 23:18 ` [PATCH v5 12/17] builtin/repack.c: support generating a cruft pack Taylor Blau
2022-05-20 23:18 ` [PATCH v5 13/17] builtin/repack.c: allow configuring cruft pack generation Taylor Blau
2022-05-20 23:18 ` [PATCH v5 14/17] builtin/repack.c: use named flags for existing_packs Taylor Blau
2022-05-20 23:18 ` [PATCH v5 15/17] builtin/repack.c: add cruft packs to MIDX during geometric repack Taylor Blau
2022-05-20 23:18 ` [PATCH v5 16/17] builtin/gc.c: conditionally avoid pruning objects via loose Taylor Blau
2022-06-19 5:38 ` René Scharfe
2022-06-21 15:58 ` Junio C Hamano
2022-05-20 23:18 ` [PATCH v5 17/17] sha1-file.c: don't freshen cruft packs Taylor Blau
2022-05-21 11:17 ` [PATCH v5 00/17] " Ævar Arnfjörð Bjarmason
2022-05-24 19:39 ` Jonathan Nieder
2022-05-24 21:50 ` Taylor Blau
2022-05-24 21:55 ` Ævar Arnfjörð Bjarmason
2022-05-24 22:12 ` Taylor Blau
2022-05-25 7:53 ` Jonathan Nieder
2022-05-25 19:59 ` Derrick Stolee
2022-05-25 21:09 ` Taylor Blau
2022-05-26 0:06 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yo6b+8sixGAqMm/x@nand.local \
--to=me@ttaylorr.com \
--cc=avarab@gmail.com \
--cc=derrickstolee@github.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jrnieder@gmail.com \
--cc=larsxschneider@gmail.com \
--cc=rsbecker@nexbridge.com \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.