Re: [PATCH v2 1/1] midx.c: fix an integer overflow

From: Derrick Stolee <stolee@gmail.com>
To: Damien Robert <damien.olivier.robert@gmail.com>,
	Jeff King <peff@peff.net>,
	git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Derrick Stolee <dstolee@microsoft.com>,
	William Baker <William.Baker@microsoft.com>
Cc: Damien Robert <damien.olivier.robert+git@gmail.com>
Subject: Re: [PATCH v2 1/1] midx.c: fix an integer overflow
Date: Thu, 12 Mar 2020 14:28:11 -0400	[thread overview]
Message-ID: <a538c497-a79d-43be-3d00-c7a619acc4e6@gmail.com> (raw)
In-Reply-To: <20200312173520.2401776-1-damien.olivier.robert+git@gmail.com>

On 3/12/2020 1:35 PM, Damien Robert wrote:
> When verifying a midx index with 0 objects, the
>     m->num_objects - 1
> overflows to 4294967295.
> 
> Fix this both by checking that the midx contains at least one oid,
> and also that we don't write any midx when there is no packfiles.
> 
> Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com>
> ---
> Should I add a test? It is a bit troublesome to generate a zero object midx
> file since this patch prevents it from using 'midx write'...

I'm glad that your patch makes it impossible to generate a zero-object
multi-pack-index, and that makes a test hard to implement. I'm not sure
what history Git has for storing explicit binary content into the test
suite. There really is only one "empty" multi-pack-index, but it is
unfortunately still a bit big for a test case to write explicitly due
to the 256-word fanout table.

I _think_ the t/tXXXX directories are used for this kind of data storage,
so you could generate an empty multi-pack-index from an older version of
Git then store it there. Please wait for someone else on-list to say that
this is a good idea, though. It may not be worth the pain of a binary file
in the patch.

>  midx.c | 35 +++++++++++++++++++++++------------
>  1 file changed, 23 insertions(+), 12 deletions(-)
> 
> diff --git a/midx.c b/midx.c
> index 1527e464a7..2cece7f9ea 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -923,6 +923,12 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
>  	cur_chunk = 0;
>  	num_chunks = large_offsets_needed ? 5 : 4;
>  
> +	if (packs.nr - dropped_packs == 0) {
> +		error(_("no pack files to index."));

nit: I would use "pack-files" here. Second best is "packfiles".

> +		result = 1;
> +		goto cleanup;
> +	}
> +
>  	written = write_midx_header(f, num_chunks, packs.nr - dropped_packs);
>  
>  	chunk_ids[cur_chunk] = MIDX_CHUNKID_PACKNAMES;
> @@ -1124,22 +1130,27 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag
>  				    i, oid_fanout1, oid_fanout2, i + 1);
>  	}
>  
> -	if (flags & MIDX_PROGRESS)
> -		progress = start_sparse_progress(_("Verifying OID order in multi-pack-index"),
> -						 m->num_objects - 1);
> -	for (i = 0; i < m->num_objects - 1; i++) {
> -		struct object_id oid1, oid2;
> +	if (m->num_objects == 0)
> +		midx_report(_("Warning: the midx contains no oid."));

Should this "Warning: " be here? The other calls to midx_report() do not have such prefix.

It could be valuable to add "warning: %s\n" to the fprintf inside midx_report(), but that should be done as its own patch.

Also, it may be valuable to return from this block so you do not need to put the block below in a tabbed block, reducing the complexity of this patch.

> +	else
> +	{
> +		if (flags & MIDX_PROGRESS)
> +			progress = start_sparse_progress(_("Verifying OID order in multi-pack-index"),
> +							 m->num_objects - 1);
> +		for (i = 0; i < m->num_objects - 1; i++) {
> +			struct object_id oid1, oid2;
>  
> -		nth_midxed_object_oid(&oid1, m, i);
> -		nth_midxed_object_oid(&oid2, m, i + 1);
> +			nth_midxed_object_oid(&oid1, m, i);
> +			nth_midxed_object_oid(&oid2, m, i + 1);
>  
> -		if (oidcmp(&oid1, &oid2) >= 0)
> -			midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"),
> -				    i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
> +			if (oidcmp(&oid1, &oid2) >= 0)
> +				midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"),
> +					    i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
>  
> -		midx_display_sparse_progress(progress, i + 1);
> +			midx_display_sparse_progress(progress, i + 1);
> +		}
> +		stop_progress(&progress);
>  	}
> -	stop_progress(&progress);
>  
>  	/*
>  	 * Create an array mapping each object to its packfile id.  Sort it
> 

Thanks for digging into this!

-Stolee