[PATCH] Support for large files on 32bit systems.

[PATCH] Support for large files on 32bit systems.

[Martin Waitz]

Glibc uses the same size for int and off_t by default.
In order to support large pack sizes (>2GB) we force Glibc to a 64bit off_t.

Signed-off-by: Martin Waitz <tali@admingilde.org>
---
 git-compat-util.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index bf3ceb8..38f9594 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -19,6 +19,11 @@
 #define _GNU_SOURCE 1
 #define _BSD_SOURCE 1
 
+/* support large files even on 32bit systems */
+#ifdef __GLIBC__
+#define _FILE_OFFSET_BITS=64
+#endif
+
 #include <unistd.h>
 #include <stdio.h>
 #include <sys/stat.h>
-- 
1.5.0.rc2.g08361-dirty

-- 
Martin Waitz

Re: [PATCH] Support for large files on 32bit systems.

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

hoi :)

arg, this patch is completely broken.
I tested it by setting the macro via the command line and the
= is still left in there.  And __GLIBC__ is not yet defined at this
point.

However, we need to set _FILE_OFFSET_BITS for glibc somehow.
What is the best way to do so?

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Support for large files on 32bit systems.

[Shawn O. Pearce]

Martin Waitz <tali@admingilde.org> wrote:
> arg, this patch is completely broken.
> I tested it by setting the macro via the command line and the
> = is still left in there.  And __GLIBC__ is not yet defined at this
> point.
> 
> However, we need to set _FILE_OFFSET_BITS for glibc somehow.
> What is the best way to do so?

I think the only way to do with this is to have the Makefile detect
if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
some rule (e.g. uname output?), then add that to CFLAGS when needed.

This is one area where autoconf is nice, as you can compile
a thousand little C programs to see how the compiler reacts by
default, and when you attempt to set this define, and then decide
if you should include it or not.

It is also the part of autoconf that's not-so-nice... as it compiles
a thousand little C programs just to figure out how to compile the
real program.  ;-)

-- 
Shawn.

Re: [PATCH] Support for large files on 32bit systems.

[Martin Waitz]

hoi :)

On Sat, Feb 17, 2007 at 04:49:59AM -0500, Shawn O. Pearce wrote:
> I think the only way to do with this is to have the Makefile detect
> if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
> some rule (e.g. uname output?), then add that to CFLAGS when needed.

something like this?

(I'm just testing it with a large data transfer, which will take some
time..., will report success later)

+++
Support for large files on 32bit systems.

Glibc uses the same size for int and off_t by default.
In order to support large pack sizes (>2GB) we force Glibc to a 64bit off_t.

Signed-off-by: Martin Waitz <tali@admingilde.org>
---
 Makefile |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index ebecbbd..325c19f 100644
--- a/Makefile
+++ b/Makefile
@@ -334,9 +334,11 @@ EXTLIBS = -lz
 
 ifeq ($(uname_S),Linux)
 	NO_STRLCPY = YesPlease
+	BASIC_CFLAGS = -D_FILE_OFFSET_BITS=64
 endif
 ifeq ($(uname_S),GNU/kFreeBSD)
 	NO_STRLCPY = YesPlease
+	BASIC_CFLAGS = -D_FILE_OFFSET_BITS=64
 endif
 ifeq ($(uname_S),Darwin)
 	NEEDS_SSL_WITH_CRYPTO = YesPlease
-- 
1.5.0.80.g42d14


-- 
Martin Waitz

Re: [PATCH] Support for large files on 32bit systems.

[Nicolas Pitre]

On Sat, 17 Feb 2007, Shawn O. Pearce wrote:

> Martin Waitz <tali@admingilde.org> wrote:
> > arg, this patch is completely broken.
> > I tested it by setting the macro via the command line and the
> > = is still left in there.  And __GLIBC__ is not yet defined at this
> > point.
> > 
> > However, we need to set _FILE_OFFSET_BITS for glibc somehow.
> > What is the best way to do so?
> 
> I think the only way to do with this is to have the Makefile detect
> if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
> some rule (e.g. uname output?), then add that to CFLAGS when needed.

Why not simply defining _FILE_OFFSET_BITS=64 unconditionally?

It certainly won't cause GIT to explode if compiled against something 
else than glibc.


Nicolas

Re: [PATCH] Support for large files on 32bit systems.

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

>> I think the only way to do with this is to have the Makefile detect
>> if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
>> some rule (e.g. uname output?), then add that to CFLAGS when needed.
>
> Why not simply defining _FILE_OFFSET_BITS=64 unconditionally?
>
> It certainly won't cause GIT to explode if compiled against something 
> else than glibc.

I like the simplicity of that approach.  Like this?

diff --git a/git-compat-util.h b/git-compat-util.h
index 105ac28..dd5f369 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1,6 +1,8 @@
 #ifndef GIT_COMPAT_UTIL_H
 #define GIT_COMPAT_UTIL_H
 
+#define _FILE_OFFSET_BITS 64
+
 #ifndef FLEX_ARRAY
 #if defined(__GNUC__) && (__GNUC__ < 3)
 #define FLEX_ARRAY 0