Re: [PATCH 3/7] commit-graph: start parsing generation v2 (again)

[Derrick Stolee <derrickstolee@github.com>]

On 3/1/2022 9:53 AM, Patrick Steinhardt wrote:
> On Tue, Mar 01, 2022 at 09:06:44AM -0500, Derrick Stolee wrote:
>> On 3/1/2022 5:35 AM, Patrick Steinhardt wrote:
>>> On Tue, Mar 01, 2022 at 10:46:14AM +0100, Patrick Steinhardt wrote:
>>>> On Mon, Feb 28, 2022 at 01:44:01PM -0500, Derrick Stolee wrote:
>>>>> On 2/28/2022 11:59 AM, Patrick Steinhardt wrote:
>>>>>> On Mon, Feb 28, 2022 at 11:23:38AM -0500, Derrick Stolee wrote:
>>>>>>> On 2/28/2022 10:18 AM, Patrick Steinhardt wrote:
>>>>>>>> [1]: https://gitlab.com/gitlab-com/www-gitlab-com.git
...
>>> So the question is whether this is a change that needs to be rolled out
>>> over multiple releases. First we'd get in the bug fix such that we write
>>> correct commit-graphs, and after this fix has been released we can also
>>> release the fix that starts to actually parse the generation. This
>>> ensures there's a grace period during which we can hopefully correct the
>>> data on-disk such that users are not faced with failures.
>>
>> You are right that we need to be careful here, but I also think that
>> previous versions of Git always wrote the correct data. Here is my
>> thought process:
>>
>> 1. To get this bug, we need to have parsed the corrected commit date
>>    from an existing commit-graph in order to under-count the number
>>    of overflow values.
>>
>> 2. Before this series, Git versions were not parsing the corrected
>>    commit date, so they recompute the corrected commit date every
>>    time the commit-graph is written, getting the proper count of
>>    overflow values.
>>
>> For these reasons, data written by previous versions of Git are
>> correct and can be trusted without a staged release.
>>
>> Does this make sense? Or, do you experience a different result when
>> you build commit-graphs with a released Git version and then when
>> writing on top with all patches applied?
> 
> Just to verify my understanding: you claim that the bug I was hitting
> shouldn't be encountered in the wild when the release , but
> only if one were to write a commit-graph with the intermediate stafe
> until patch 3/4 of your patch series?

That is my claim. And my testing of the repo at [1] has demonstrated
that it works correctly in these cases.
 
> Hum. I have re-verified, and this indeed seems to play out. So I must've
> accidentally ran all my testing with the state generated without the
> final patch which fixes the corruption. I do see lots of the following
> warnings, but overall I can verify and write the commit-graph just fine:
> 
>     commit-graph generation for commit c80a42de8803e2d77818d0c82f88e748d7f9425f is 1623362063 < 1623362139

But I'm not able to generate these warnings from either version. I
tried generating different levels of a split commit-graph, but
could not reproduce it. If you have reproduction steps using current
'master' (or any released Git version) and the four patches here,
then I would love to get a full understanding of your errors.

Thanks,
-Stolee