Re: adding new 32-bit on-disk (unsigned) timestamp formats (was: [PATCH v5 02/17] pack-mtimes: support reading .mtimes files)

[Taylor Blau <me@ttaylorr.com>]

On Thu, May 26, 2022 at 02:02:39AM +0200, Ævar Arnfjörð Bjarmason wrote:
> >> > If we really are trying to micro-optimize storage space here I'm willing
> >> > to bet that this is still a bad/premature optimization. There's much
> >> > better ways to store this sort of data in a compact way if that's the
> >> > concern. E.g. you'd store a 64 bit "base" timestamp in the header for
> >> > the first entry, and have smaller (signed) "delta" timestamps storing
> >> > offsets from that "base" timestamp.
> >>
> >> This is a good idea for a v2 format when that is necessary.
> >
> > I agree here.
> >
> > I'm not opposed to such a change (or even being the one to work on it!),
> > but I would encourage us to pursue that change outside of this series,
> > since it can easily be done on top.
> >
> > Of course, if we ever did decide to implement 64-bit mtimes, we would
> > have to maintain support for reading both the 32-bit and 64-bit values.
> > But I think the code is well-equipped to do that, and it could be done
> > on top without significant additional complexity.
>
> Do you mean "on top" in the sense that we'd expect that before the next
> release, so that we wouldn't need to deal with bumping the format, and
> have some phase-out period for the older version etc.
>
> Or that we would need to treat what's landing here as something we'll
> need to support going forward?

My plan is to treat what will hopefully land here as something we're
going to support.

I meant "on top" in the sense that the format implemented here does not
restrict us against making changes (like adding support for wider
records) in the future. IOW, I did not mean to suggest that we should
expect more patches from me in this cycle to deprecate parts of the v1
format.

In other words (again ;-)), I would like to see us ship this format with
the existing 32-bit records.

> I think if a format change is worthwhile doing at all that it's worth
> just doing it now if it's going to be the latter of those, as changing
> file formats before they're in the wild is easy, but after that it's at
> best a bit tedious. E.g. we'll need testing to see how we deal with
> mixed new/old format files etc. etc.

I can understand where you're coming from, though as I noted earlier in
the thread, I don't think changing the format in the manner you suggest
would be that difficult in practice.

But in the meantime, the existing format is useful and works, and I
don't think we should go back to the drawing board for something that we
can do later if we decide to.

Thanks,
Taylor