git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org,
	"Sibi Siddharthan" <sibisiddharthan.github@gmail.com>,
	"Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
	"SZEDER Gábor" <szeder.dev@gmail.com>,
	"Johannes Schindelin" <johannes.schindelin@gmx.de>
Subject: Re: [PATCH 10/10] hashmap_for_each_entry(): work around MSVC's run-time check failure #3
Date: Sat, 26 Sep 2020 09:54:13 -0700	[thread overview]
Message-ID: <xmqqy2kwiimi.fsf@gitster.c.googlers.com> (raw)
In-Reply-To: <xmqq8scxln10.fsf@gitster.c.googlers.com> (Junio C. Hamano's message of "Fri, 25 Sep 2020 11:38:35 -0700")

Junio C Hamano <gitster@pobox.com> writes:

> Whoa, wait.  If it is just that macro, can we perhaps do something
> like the attached patch?

I looked at all the uses of OFFSETOF_VAR() and I think the one used
for hashmap_for_each_entry() is the only instance that 'var' given
to it can legitimately be uninitialized, if typeof() were available.

Here are the findings.

#define hashmap_put_entry(map, keyvar, member) \
	container_of_or_null_offset(hashmap_put(map, &(keyvar)->member), \
				OFFSETOF_VAR(keyvar, member))

The keyvar is a pointer to the entry being placed in the map; it
must hold a valid one so the pointer-diff implementation of
OFFSETOF_VAR() should work fine, or we are putting garbage in to the
map.

#define hashmap_remove_entry(map, keyvar, member, keydata) \
	container_of_or_null_offset( \
			hashmap_remove(map, &(keyvar)->member, keydata), \
			OFFSETOF_VAR(keyvar, member))

The keyvar is used to match against an existing entry in the map to
be removed---it must have a valid value.

#define hashmap_for_each_entry(map, iter, var, member) \
	for (var = hashmap_iter_first_entry_offset(map, iter, \
						OFFSETOF_VAR(var, member)); \
		var; \
		var = hashmap_iter_next_entry_offset(iter, \
						OFFSETOF_VAR(var, member)))

This, as you discovered, can be fed an uninitialized var and the
first thing it does is to use OFFSETOF_VAR() on it in order to call
hashmap_iter_first_entry_offset().  After that, i.e. when we called
that function to start the loop, var is defined and we would be OK.

The trick I suggested is to initialize var to NULL before making the
call to hashmap_iter_first_entry_offset(), i.e.

	for (var = NULL, \
	     var = hashmap_iter_first_entry_offset(map, iter, \
						OFFSETOF_VAR(var, member)); \

#define hashmap_get_entry(map, keyvar, member, keydata) \
	container_of_or_null_offset( \
				hashmap_get(map, &(keyvar)->member, keydata), \
				OFFSETOF_VAR(keyvar, member))

Must be OK for the same reason _put_entry() is OK.

#define hashmap_get_next_entry(map, var, member) \
	container_of_or_null_offset(hashmap_get_next(map, &(var)->member), \
				OFFSETOF_VAR(var, member))

This tries to go to the next-equal-pointer starting from var, so var
must be valid already.

So, perhaps the attached may be a viable replacement that would be
more futureproof with less maintenance cost, I suspect.

Thanks.

--- >8 ----- cut here ----- >8 ---
Subject: hashmap_for_each_entry(): workaround MSVC's runtime check failure #3

The OFFSETOF_VAR(var, member) macro is implemented in terms of
offsetof(typeof(*var), member) with compilers that know typeof(),
but its fallback implemenation compares &(var->member) and (var) and
count the distance in bytes, i.e.

    ((uintptr_t)&(var)->member - (uintptr_t)(var))

MSVC's runtime check, when fed an uninitialized 'var', flags this as
a use of an uninitialized variable (and that is legit---uninitialized
contents of 'var' is subtracted) in a debug build.

After auditing all 6 uses of OFFSETOF_VAR(), 1 of them does feed a
potentially uninitialized 'var' to the macro in the beginning of the
for() loop:

    #define hashmap_for_each_entry(map, iter, var, member) \
            for (var = hashmap_iter_first_entry_offset(map, iter, \
                                                    OFFSETOF_VAR(var, member)); \
                    var; \
                    var = hashmap_iter_next_entry_offset(iter, \
                                                    OFFSETOF_VAR(var, member)))

We can work around this by making sure that var has _some_ value
when OFFSETOF_VAR() is called.  Strictly speaking, it invites
undefined behaviour to use NULL here if we end up with pointer
comparison, but MSVC runtime seems to be happy with it, and most
other systems have typeof() and don't even need pointer comparison
fallback code.

---
 hashmap.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git c/hashmap.h w/hashmap.h
index ef220de4c6..b011b394fe 100644
--- c/hashmap.h
+++ w/hashmap.h
@@ -449,7 +449,8 @@ static inline struct hashmap_entry *hashmap_iter_first(struct hashmap *map,
  * containing a @member which is a "struct hashmap_entry"
  */
 #define hashmap_for_each_entry(map, iter, var, member) \
-	for (var = hashmap_iter_first_entry_offset(map, iter, \
+	for (var = NULL, /* for systems without typeof */ \
+	     var = hashmap_iter_first_entry_offset(map, iter, \
 						OFFSETOF_VAR(var, member)); \
 		var; \
 		var = hashmap_iter_next_entry_offset(iter, \

  reply	other threads:[~2020-09-26 16:54 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-25 14:28 [PATCH 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-25 16:25   ` Sibi Siddharthan
2020-09-26 20:32     ` Johannes Schindelin
2020-09-27  2:25       ` Đoàn Trần Công Danh
2020-09-28 13:56         ` Johannes Schindelin
2020-09-29 14:04           ` Đoàn Trần Công Danh
2020-09-29 18:42             ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-25 14:28 ` [PATCH 06/10] cmake (Windows): let the `.dll` files are found when running the tests Johannes Schindelin via GitGitGadget
2020-09-25 19:48   ` Eric Sunshine
2020-09-26 21:00     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-25 17:29   ` Sibi Siddharthan
2020-09-26 20:33     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-30  5:05   ` Sibi Siddharthan
2020-09-30 15:25     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-25 18:22   ` Junio C Hamano
2020-09-26 20:45     ` Johannes Schindelin
2020-09-25 14:28 ` [PATCH 10/10] hashmap_for_each_entry(): work around MSVC's run-time check failure #3 Johannes Schindelin via GitGitGadget
2020-09-25 18:38   ` Junio C Hamano
2020-09-26 16:54     ` Junio C Hamano [this message]
2020-09-26 20:57       ` Johannes Schindelin
2020-09-26 21:32 ` [PATCH v2 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-28 11:17     ` Øystein Walle
2020-09-28 19:39       ` Johannes Schindelin
2020-09-26 21:32   ` [PATCH v2 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 06/10] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-26 21:32   ` [PATCH v2 10/10] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
2020-09-28 21:09   ` [PATCH v3 00/11] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 01/11] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 02/11] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 03/11] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 04/11] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 05/11] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 06/11] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 07/11] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 08/11] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-29  6:51       ` Sibi Siddharthan
2020-09-29 12:07         ` Johannes Schindelin
2020-09-28 21:09     ` [PATCH v3 09/11] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 10/11] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget
2020-09-28 21:09     ` [PATCH v3 11/11] cmake: fix typo in message when `msgfmt` was not found Johannes Schindelin via GitGitGadget
2020-09-28 22:11       ` Junio C Hamano
2020-09-29 12:07         ` Johannes Schindelin
2020-09-30 15:26     ` [PATCH v4 00/10] CMake and Visual Studio Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 01/10] cmake: ignore files generated by CMake as run in " Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 02/10] cmake: do find Git for Windows' shell interpreter Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 03/10] cmake: ensure that the `vcpkg` packages are found on Windows Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 04/10] cmake: fall back to using `vcpkg`'s `msgfmt.exe` " Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 05/10] cmake: quote the path accurately when editing `test-lib.sh` Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 06/10] cmake (Windows): let the `.dll` files be found when running the tests Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 07/10] cmake (Windows): complain when encountering an unknown compiler Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 08/10] cmake (Windows): initialize vcpkg/build dependencies automatically Johannes Schindelin via GitGitGadget
2020-09-30 19:17         ` Johannes Schindelin
2020-09-30 23:08           ` Junio C Hamano
2020-09-30 15:26       ` [PATCH v4 09/10] cmake (Windows): recommend using Visual Studio's built-in CMake support Johannes Schindelin via GitGitGadget
2020-09-30 15:26       ` [PATCH v4 10/10] hashmap_for_each_entry(): workaround MSVC's runtime check failure #3 Junio C Hamano via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqy2kwiimi.fsf@gitster.c.googlers.com \
    --to=gitster@pobox.com \
    --cc=congdanhqx@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=johannes.schindelin@gmx.de \
    --cc=sibisiddharthan.github@gmail.com \
    --cc=szeder.dev@gmail.com \
    --subject='Re: [PATCH 10/10] hashmap_for_each_entry(): work around MSVC'\''s run-time check failure #3' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).