Re: [PATCH 04/14] Set _O_BINARY as default fmode for both MinGW and MSVC

[Alexey Borzenkov <snaury@gmail.com>]

GMail ate the bottom half of my message... again. :( Seems like
there's a strange bug in copy/pasting, I better compose long emails in
TextMate from now on. Here's a "reconstruction":

On Tue, Sep 15, 2009 at 5:44 PM, Marius Storm-Olsen <mstormo@gmail.com> wrote:
> +extern int _fmode;

And indeed. I just ported this patch to my custom msysgit branch
(based on v1.6.4.3) and it didn't compile:

   CC git.o
cc1.exe: warnings being treated as errors
In file included from git-compat-util.h:116,
                from builtin.h:4,
                from git.c:1:
compat/mingw.h:243: error: '_fmode' redeclared without dllimport
attribute: previous dllimport ignored
git.c: In function 'main':
git.c:456: error: the address of '_iob' will always evaluate as 'true'
make: *** [git.o] Error 1

> +       if (stdin) \
> +               _setmode(_fileno(stdin), _O_BINARY); \
> +       if (stdout) \
> +               _setmode(_fileno(stdout), _O_BINARY); \
> +       if (stderr) \
> +               _setmode(_fileno(stderr), _O_BINARY); \

Also, at least mingw/gcc that is coming with msysgit thinks that
stdin/stdout/stderr always evaluate to true, and this check causes
problems as well. In the end, your patch should become something like
this:

diff --git a/compat/mingw.c b/compat/mingw.c
index fd642e4..807996c 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -5,7 +5,6 @@
 #include "../strbuf.h"

 extern int hide_dotfiles;
-unsigned int _CRT_fmode = _O_BINARY;

 static int err_win_to_posix(DWORD winerr)
 {
diff --git a/compat/mingw.h b/compat/mingw.h
index cfbcc0e..46473c5 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -244,6 +244,10 @@ char **env_setenv(char **env, const char *name);
 static int mingw_main(); \
 int main(int argc, const char **argv) \
 { \
+	_fmode = _O_BINARY; \
+	_setmode(_fileno(stdin), _O_BINARY); \
+	_setmode(_fileno(stdout), _O_BINARY); \
+	_setmode(_fileno(stderr), _O_BINARY); \
 	argv[0] = xstrdup(_pgmptr); \
 	return mingw_main(argc, argv); \
 } \

I can't check if it compiles with MSVC, but with msysgit it compiles
fine. Tests pass at least up to 3400 (haven't finished the rest).