From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To: Junio C Hamano <gitster@pobox.com>
Cc: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>,
git@vger.kernel.org,
"Sibi Siddharthan" <sibisiddharthan.github@gmail.com>,
"Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
"SZEDER Gábor" <szeder.dev@gmail.com>
Subject: Re: [PATCH 10/10] hashmap_for_each_entry(): work around MSVC's run-time check failure #3
Date: Sat, 26 Sep 2020 22:57:09 +0200 (CEST) [thread overview]
Message-ID: <nycvar.QRO.7.76.6.2009262254310.50@tvgsbejvaqbjf.bet> (raw)
In-Reply-To: <xmqqy2kwiimi.fsf@gitster.c.googlers.com>
Hi Junio,
On Sat, 26 Sep 2020, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Whoa, wait. If it is just that macro, can we perhaps do something
> > like the attached patch?
>
> I looked at all the uses of OFFSETOF_VAR() and I think the one used
> for hashmap_for_each_entry() is the only instance that 'var' given
> to it can legitimately be uninitialized, if typeof() were available.
Thank you for doing all that leg work. TBH I didn't even think about
looking further, after having run a couple tests manually that I thought
were exhaustive in exercising this type of code pattern.
> Here are the findings.
>
> #define hashmap_put_entry(map, keyvar, member) \
> container_of_or_null_offset(hashmap_put(map, &(keyvar)->member), \
> OFFSETOF_VAR(keyvar, member))
>
> The keyvar is a pointer to the entry being placed in the map; it
> must hold a valid one so the pointer-diff implementation of
> OFFSETOF_VAR() should work fine, or we are putting garbage in to the
> map.
>
> #define hashmap_remove_entry(map, keyvar, member, keydata) \
> container_of_or_null_offset( \
> hashmap_remove(map, &(keyvar)->member, keydata), \
> OFFSETOF_VAR(keyvar, member))
>
> The keyvar is used to match against an existing entry in the map to
> be removed---it must have a valid value.
>
> #define hashmap_for_each_entry(map, iter, var, member) \
> for (var = hashmap_iter_first_entry_offset(map, iter, \
> OFFSETOF_VAR(var, member)); \
> var; \
> var = hashmap_iter_next_entry_offset(iter, \
> OFFSETOF_VAR(var, member)))
>
> This, as you discovered, can be fed an uninitialized var and the
> first thing it does is to use OFFSETOF_VAR() on it in order to call
> hashmap_iter_first_entry_offset(). After that, i.e. when we called
> that function to start the loop, var is defined and we would be OK.
>
> The trick I suggested is to initialize var to NULL before making the
> call to hashmap_iter_first_entry_offset(), i.e.
>
> for (var = NULL, \
> var = hashmap_iter_first_entry_offset(map, iter, \
> OFFSETOF_VAR(var, member)); \
>
> #define hashmap_get_entry(map, keyvar, member, keydata) \
> container_of_or_null_offset( \
> hashmap_get(map, &(keyvar)->member, keydata), \
> OFFSETOF_VAR(keyvar, member))
>
> Must be OK for the same reason _put_entry() is OK.
>
> #define hashmap_get_next_entry(map, var, member) \
> container_of_or_null_offset(hashmap_get_next(map, &(var)->member), \
> OFFSETOF_VAR(var, member))
>
> This tries to go to the next-equal-pointer starting from var, so var
> must be valid already.
>
> So, perhaps the attached may be a viable replacement that would be
> more futureproof with less maintenance cost, I suspect.
Definitely much nicer to maintain, and easier to verify. In my hands, this
works better than my manual touch-ups of _all_ the call sites. So I
replaced my patch with yours (adding your SOB).
Ciao,
Dscho
> Thanks.
>
> --- >8 ----- cut here ----- >8 ---
> Subject: hashmap_for_each_entry(): workaround MSVC's runtime check failure #3
>
> The OFFSETOF_VAR(var, member) macro is implemented in terms of
> offsetof(typeof(*var), member) with compilers that know typeof(),
> but its fallback implemenation compares &(var->member) and (var) and
> count the distance in bytes, i.e.
>
> ((uintptr_t)&(var)->member - (uintptr_t)(var))
>
> MSVC's runtime check, when fed an uninitialized 'var', flags this as
> a use of an uninitialized variable (and that is legit---uninitialized
> contents of 'var' is subtracted) in a debug build.
>
> After auditing all 6 uses of OFFSETOF_VAR(), 1 of them does feed a
> potentially uninitialized 'var' to the macro in the beginning of the
> for() loop:
>
> #define hashmap_for_each_entry(map, iter, var, member) \
> for (var = hashmap_iter_first_entry_offset(map, iter, \
> OFFSETOF_VAR(var, member)); \
> var; \
> var = hashmap_iter_next_entry_offset(iter, \
> OFFSETOF_VAR(var, member)))
>
> We can work around this by making sure that var has _some_ value
> when OFFSETOF_VAR() is called. Strictly speaking, it invites
> undefined behaviour to use NULL here if we end up with pointer
> comparison, but MSVC runtime seems to be happy with it, and most
> other systems have typeof() and don't even need pointer comparison
> fallback code.
>
> ---
> hashmap.h | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git c/hashmap.h w/hashmap.h
> index ef220de4c6..b011b394fe 100644
> --- c/hashmap.h
> +++ w/hashmap.h
> @@ -449,7 +449,8 @@ static inline struct hashmap_entry *hashmap_iter_first(struct hashmap *map,
> * containing a @member which is a "struct hashmap_entry"
> */
> #define hashmap_for_each_entry(map, iter, var, member) \
> - for (var = hashmap_iter_first_entry_offset(map, iter, \
> + for (var = NULL, /* for systems without typeof */ \
> + var = hashmap_iter_first_entry_offset(map, iter, \
> OFFSETOF_VAR(var, member)); \
> var; \
> var = hashmap_iter_next_entry_offset(iter, \
>
>
next prev parent reply other threads:[~2020-09-26 20:57 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 14:28 [PATCH 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-25 16:25 ` Sibi Siddharthan
2020-09-26 20:32 ` Johannes Schindelin
2020-09-27 2:25 ` Đoàn Trần Công Danh
2020-09-28 13:56 ` Johannes Schindelin
2020-09-29 14:04 ` Đoàn Trần Công Danh
2020-09-29 18:42 ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 06/10] cmake (Windows): let the `.dll` files are found when running the tests Johannes Schindelin via GitGitGadget
2020-09-25 19:48 ` Eric Sunshine
2020-09-26 21:00 ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-25 17:29 ` Sibi Siddharthan
2020-09-26 20:33 ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-30 5:05 ` Sibi Siddharthan
2020-09-30 15:25 ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-25 18:22 ` Junio C Hamano
2020-09-26 20:45 ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 10/10] hashmap_for_each_entry(): work around MSVC's run-time check failure #3 Johannes Schindelin via GitGitGadget
2020-09-25 18:38 ` Junio C Hamano
2020-09-26 16:54 ` Junio C Hamano
2020-09-26 20:57 ` Johannes Schindelin [this message]
2020-09-26 21:32 ` [PATCH v2 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-28 11:17 ` Øystein Walle
2020-09-28 19:39 ` Johannes Schindelin
2020-09-26 21:32 ` [PATCH v2 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 06/10] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-26 21:32 ` [PATCH v2 10/10] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 00/11] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 01/11] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 02/11] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 03/11] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 04/11] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 05/11] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 06/11] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 07/11] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 08/11] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-29 6:51 ` Sibi Siddharthan
2020-09-29 12:07 ` Johannes Schindelin
2020-09-28 21:09 ` [PATCH v3 09/11] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 10/11] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
2020-09-28 21:09 ` [PATCH v3 11/11] cmake: fix typo in message when `msgfmt` was not found Johannes Schindelin via GitGitGadget
2020-09-28 22:11 ` Junio C Hamano
2020-09-29 12:07 ` Johannes Schindelin
2020-09-30 15:26 ` [PATCH v4 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 06/10] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-30 19:17 ` Johannes Schindelin
2020-09-30 23:08 ` Junio C Hamano
2020-09-30 15:26 ` [PATCH v4 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-30 15:26 ` [PATCH v4 10/10] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=nycvar.QRO.7.76.6.2009262254310.50@tvgsbejvaqbjf.bet \
--to=johannes.schindelin@gmx.de \
--cc=congdanhqx@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=gitster@pobox.com \
--cc=sibisiddharthan.github@gmail.com \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).