Re: [PATCH v4 00/29] tree-walk: mostly replace "mode" with "enum object_type"

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Sun, Mar 21 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> Yes, that would be a serious regression. I agree that all these
>> functions/callbacks etc. should have a way to get at the mode bits.
>>
>> I'm adding "enum object_type", not removing the "mode" parameter in
>> read_tree_fn_t. This function (which is in "seen" as 03316f20347
>> (sparse-index: implement ensure_full_index(), 2021-03-16)) works just
>> fine in combination with this series.
>>
>> The other APIs modified here all retain the ability to give you the mode
>> bit, they (the tree-walk.h changes) just optionally give you the option
>> of getting just the type (or just the path), and as it turns out most
>> users of the API can be converted over to that.
>
> I have a vague feeling that such an approach may be still repeating
> the same mistake.
>
> If the original premise is that "unsigned mode" bit can be abused to
> feed impossible values like 0100653 to the system and the code
> should catch it, ...
>
>> The current codebase will allow you to stick arbitrary mode bits in
>> trees, ...
>
> ... then I would understand it if the approach is to introduce a
> distinct type that enumerates all possible mode bit values (and
> nothing else), and have the compiler validate the callchain, so that
> nobody can pass bogus mode bits (and when reading mode bits fields
> in existing objects, the one that converts from the series of octal
> bytes to that distsinct type would notice and complain).  And we've
> already seen from this particular breakages that the "distinct type"
> appropriate to be used for that purpose is not "enum object_type".
> It is not expressive enough to enumerate all possible mode bit
> values (besides, an enum is interchangeable with an int, so there
> isn't much protection we would be getting from the compiler---we
> could use a small struct of a new type, and have one static const
> instance for each possible mode bit combination, I guess, but the
> point here is that insisting on using "enum object_type" seems to be
> the source of the problem).

Yes, this is a good suggestion, but one which'll be much easier to do
after this series. Since the majority of callers don't care about the
raw mode bits or anything except if the entry is a blob/tree/commit.

> I am afraid that it is even worse to pass both object type and
> "unsigned mode" together.  It would still leave room for a bug to
> pass nonsense mode bits, which we said we wanted to catch in our
> original mission statement.  In addition, we now have a new room for
> a bug, which is to pass an inconsistent pair of object type and mode
> to the callchain.  Somebody would need to say "yuck, we got a mode
> 100644 but the type says TREE" now, in addition to validating if the
> mode is sensible, which we should be doing somehow, no?

Indeed. This goes back to my "bear with me" comment upthread. I'm
planning to fix these cases too, and starting by moving most things to
"enum object_type" makes that a lot easier.

Right now we simply trust the mode bits, but that opens the door to the
same sort of bug I'm fixing in another series (that I need to re-roll)
where we have a "type commit" and "object ID" in a tag, but the ID is
really a tree or whatever.

We then get confused because we trusted the invalid metadata over
checking the type of the object ID we have in the object store.

So this particular series doesn't fix this bug at the end of it, the
"object_type" in tree-walk.c is just a function of deriving it from the
mode bits.

But once I've audited the cases where we're really just acting on the
type and don't have or use the mode (so we know we're not re-writing the
mode somewhere) then we would set the object_type to the actual type, as
in what we'd get from oid_object_info(ID).

Then we can e.g. scream murder about a mode/type mismatch in fsck, but
still be able to list/diff/inspect etc. that object in any codepath that
just wants to e.g. recursively iterate over the tree.



> So, I am not sure how these changes are making anything better.
>
>> ... I had a
>> summary of this in v1, but should probably have provided a recap[1].
>
> Oh, absolutely.  When we iterate, we should be welcoming to those
> who missed earlier iterations.