git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To: Junio C Hamano <gitster@pobox.com>
Cc: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>,
	git@vger.kernel.org,
	"Sibi Siddharthan" <sibisiddharthan.github@gmail.com>,
	"Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
	"SZEDER Gábor" <szeder.dev@gmail.com>
Subject: Re: [PATCH 10/10] hashmap_for_each_entry(): work around MSVC's run-time check failure #3
Date: Sat, 26 Sep 2020 22:57:09 +0200 (CEST)	[thread overview]
Message-ID: <nycvar.QRO.7.76.6.2009262254310.50@tvgsbejvaqbjf.bet> (raw)
In-Reply-To: <xmqqy2kwiimi.fsf@gitster.c.googlers.com>

Hi Junio,

On Sat, 26 Sep 2020, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
>
> > Whoa, wait.  If it is just that macro, can we perhaps do something
> > like the attached patch?
>
> I looked at all the uses of OFFSETOF_VAR() and I think the one used
> for hashmap_for_each_entry() is the only instance that 'var' given
> to it can legitimately be uninitialized, if typeof() were available.

Thank you for doing all that leg work. TBH I didn't even think about
looking further, after having run a couple tests manually that I thought
were exhaustive in exercising this type of code pattern.

> Here are the findings.
>
> #define hashmap_put_entry(map, keyvar, member) \
> 	container_of_or_null_offset(hashmap_put(map, &(keyvar)->member), \
> 				OFFSETOF_VAR(keyvar, member))
>
> The keyvar is a pointer to the entry being placed in the map; it
> must hold a valid one so the pointer-diff implementation of
> OFFSETOF_VAR() should work fine, or we are putting garbage in to the
> map.
>
> #define hashmap_remove_entry(map, keyvar, member, keydata) \
> 	container_of_or_null_offset( \
> 			hashmap_remove(map, &(keyvar)->member, keydata), \
> 			OFFSETOF_VAR(keyvar, member))
>
> The keyvar is used to match against an existing entry in the map to
> be removed---it must have a valid value.
>
> #define hashmap_for_each_entry(map, iter, var, member) \
> 	for (var = hashmap_iter_first_entry_offset(map, iter, \
> 						OFFSETOF_VAR(var, member)); \
> 		var; \
> 		var = hashmap_iter_next_entry_offset(iter, \
> 						OFFSETOF_VAR(var, member)))
>
> This, as you discovered, can be fed an uninitialized var and the
> first thing it does is to use OFFSETOF_VAR() on it in order to call
> hashmap_iter_first_entry_offset().  After that, i.e. when we called
> that function to start the loop, var is defined and we would be OK.
>
> The trick I suggested is to initialize var to NULL before making the
> call to hashmap_iter_first_entry_offset(), i.e.
>
> 	for (var = NULL, \
> 	     var = hashmap_iter_first_entry_offset(map, iter, \
> 						OFFSETOF_VAR(var, member)); \
>
> #define hashmap_get_entry(map, keyvar, member, keydata) \
> 	container_of_or_null_offset( \
> 				hashmap_get(map, &(keyvar)->member, keydata), \
> 				OFFSETOF_VAR(keyvar, member))
>
> Must be OK for the same reason _put_entry() is OK.
>
> #define hashmap_get_next_entry(map, var, member) \
> 	container_of_or_null_offset(hashmap_get_next(map, &(var)->member), \
> 				OFFSETOF_VAR(var, member))
>
> This tries to go to the next-equal-pointer starting from var, so var
> must be valid already.
>
> So, perhaps the attached may be a viable replacement that would be
> more futureproof with less maintenance cost, I suspect.

Definitely much nicer to maintain, and easier to verify. In my hands, this
works better than my manual touch-ups of _all_ the call sites. So I
replaced my patch with yours (adding your SOB).

Ciao,
Dscho

> Thanks.
>
> --- >8 ----- cut here ----- >8 ---
> Subject: hashmap_for_each_entry(): workaround MSVC's runtime check failure #3
>
> The OFFSETOF_VAR(var, member) macro is implemented in terms of
> offsetof(typeof(*var), member) with compilers that know typeof(),
> but its fallback implemenation compares &(var->member) and (var) and
> count the distance in bytes, i.e.
>
>     ((uintptr_t)&(var)->member - (uintptr_t)(var))
>
> MSVC's runtime check, when fed an uninitialized 'var', flags this as
> a use of an uninitialized variable (and that is legit---uninitialized
> contents of 'var' is subtracted) in a debug build.
>
> After auditing all 6 uses of OFFSETOF_VAR(), 1 of them does feed a
> potentially uninitialized 'var' to the macro in the beginning of the
> for() loop:
>
>     #define hashmap_for_each_entry(map, iter, var, member) \
>             for (var = hashmap_iter_first_entry_offset(map, iter, \
>                                                     OFFSETOF_VAR(var, member)); \
>                     var; \
>                     var = hashmap_iter_next_entry_offset(iter, \
>                                                     OFFSETOF_VAR(var, member)))
>
> We can work around this by making sure that var has _some_ value
> when OFFSETOF_VAR() is called.  Strictly speaking, it invites
> undefined behaviour to use NULL here if we end up with pointer
> comparison, but MSVC runtime seems to be happy with it, and most
> other systems have typeof() and don't even need pointer comparison
> fallback code.
>
> ---
>  hashmap.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git c/hashmap.h w/hashmap.h
> index ef220de4c6..b011b394fe 100644
> --- c/hashmap.h
> +++ w/hashmap.h
> @@ -449,7 +449,8 @@ static inline struct hashmap_entry *hashmap_iter_first(struct hashmap *map,
>   * containing a @member which is a "struct hashmap_entry"
>   */
>  #define hashmap_for_each_entry(map, iter, var, member) \
> -	for (var = hashmap_iter_first_entry_offset(map, iter, \
> +	for (var = NULL, /* for systems without typeof */ \
> +	     var = hashmap_iter_first_entry_offset(map, iter, \
>  						OFFSETOF_VAR(var, member)); \
>  		var; \
>  		var = hashmap_iter_next_entry_offset(iter, \
>
>

  reply	other threads:[~2020-09-26 20:57 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-25 14:28 [PATCH 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-25 16:25   ` Sibi Siddharthan
2020-09-26 20:32     ` Johannes Schindelin
2020-09-27  2:25       ` Đoàn Trần Công Danh
2020-09-28 13:56         ` Johannes Schindelin
2020-09-29 14:04           ` Đoàn Trần Công Danh
2020-09-29 18:42             ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 06/10] cmake (Windows): let the `.dll` files are found when running the tests Johannes Schindelin via GitGitGadget
2020-09-25 19:48   ` Eric Sunshine
2020-09-26 21:00     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-25 17:29   ` Sibi Siddharthan
2020-09-26 20:33     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-30  5:05   ` Sibi Siddharthan
2020-09-30 15:25     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-25 18:22   ` Junio C Hamano
2020-09-26 20:45     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 10/10] hashmap_for_each_entry(): work around MSVC's run-time check failure #3 Johannes Schindelin via GitGitGadget
2020-09-25 18:38   ` Junio C Hamano
2020-09-26 16:54     ` Junio C Hamano
2020-09-26 20:57       ` Johannes Schindelin [this message]
2020-09-26 21:32 ` [PATCH v2 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-28 11:17     ` Øystein Walle
2020-09-28 19:39       ` Johannes Schindelin
2020-09-26 21:32   ` [PATCH v2 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 06/10] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 10/10] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
2020-09-28 21:09   ` [PATCH v3 00/11] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 01/11] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 02/11] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 03/11] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 04/11] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 05/11] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 06/11] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 07/11] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 08/11] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-29  6:51       ` Sibi Siddharthan
2020-09-29 12:07         ` Johannes Schindelin
2020-09-28 21:09     ` [PATCH v3 09/11] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 10/11] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 11/11] cmake: fix typo in message when `msgfmt` was not found Johannes Schindelin via GitGitGadget
2020-09-28 22:11       ` Junio C Hamano
2020-09-29 12:07         ` Johannes Schindelin
2020-09-30 15:26     ` [PATCH v4 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 06/10] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-30 19:17         ` Johannes Schindelin
2020-09-30 23:08           ` Junio C Hamano
2020-09-30 15:26       ` [PATCH v4 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 10/10] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=nycvar.QRO.7.76.6.2009262254310.50@tvgsbejvaqbjf.bet \
    --to=johannes.schindelin@gmx.de \
    --cc=congdanhqx@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=gitster@pobox.com \
    --cc=sibisiddharthan.github@gmail.com \
    --cc=szeder.dev@gmail.com \
    --subject='Re: [PATCH 10/10] hashmap_for_each_entry(): work around MSVC'\''s run-time check failure #3' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).