[PATCH] pack-objects: proper pack time stamping with --max-pack-size

[PATCH] pack-objects: proper pack time stamping with --max-pack-size

[Nicolas Pitre]

Runtime pack access is done in the pack file mtime order since recent 
packs are more likely to contain frequently used objects than old packs.
However the --max-pack-size option can produce multiple packs with mtime 
in the reversed order as newer objects are always written first.

Let's modify mtime of later pack files (when any) so they appear older 
than preceding ones when a repack creates multiple packs.

Signed-off-by: Nicolas Pitre <nico@cam.org>
---
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index f504cff..4c2ed70 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -17,6 +17,8 @@
 #include "progress.h"
 #include "refs.h"
 
+#include <utime.h>
+
 #ifdef THREADED_DELTA_SEARCH
 #include "thread-utils.h"
 #include <pthread.h>
@@ -454,6 +456,7 @@ static void write_pack_file(void)
 	struct pack_header hdr;
 	int do_progress = progress >> pack_to_stdout;
 	uint32_t nr_remaining = nr_result;
+	time_t last_mtime = 0;
 
 	if (do_progress)
 		progress_state = start_progress("Writing objects", nr_result);
@@ -504,6 +507,7 @@ static void write_pack_file(void)
 
 		if (!pack_to_stdout) {
 			mode_t mode = umask(0);
+			struct stat st;
 			char *idx_tmp_name, tmpname[PATH_MAX];
 
 			umask(mode);
@@ -511,6 +515,7 @@ static void write_pack_file(void)
 
 			idx_tmp_name = write_idx_file(NULL, written_list,
 						      nr_written, sha1);
+
 			snprintf(tmpname, sizeof(tmpname), "%s-%s.pack",
 				 base_name, sha1_to_hex(sha1));
 			if (adjust_perm(pack_tmp_name, mode))
@@ -519,6 +524,28 @@ static void write_pack_file(void)
 			if (rename(pack_tmp_name, tmpname))
 				die("unable to rename temporary pack file: %s",
 				    strerror(errno));
+
+			/*
+			 * Packs are runtime accessed in their mtime
+			 * order since newer packs are more likely to contain
+			 * younger objects.  So if we are creating multiple
+			 * packs then we should modify the mtime of later ones
+			 * to preserve this property.
+			 */
+			if (stat(tmpname, &st) < 0) {
+				warning("failed to stat %s: %s",
+					tmpname, strerror(errno));
+			} else if (!last_mtime) {
+				last_mtime = st.st_mtime;
+			} else {
+				struct utimbuf utb;
+				utb.actime = st.st_atime;
+				utb.modtime = --last_mtime;
+				if (utime(tmpname, &utb) < 0)
+					warning("failed utime() on %s: %s",
+						tmpname, strerror(errno));
+			}
+
 			snprintf(tmpname, sizeof(tmpname), "%s-%s.idx",
 				 base_name, sha1_to_hex(sha1));
 			if (adjust_perm(idx_tmp_name, mode))
@@ -527,6 +554,7 @@ static void write_pack_file(void)
 			if (rename(idx_tmp_name, tmpname))
 				die("unable to rename temporary index file: %s",
 				    strerror(errno));
+
 			free(idx_tmp_name);
 			free(pack_tmp_name);
 			puts(sha1_to_hex(sha1));

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> Runtime pack access is done in the pack file mtime order since recent 
> packs are more likely to contain frequently used objects than old packs.
> However the --max-pack-size option can produce multiple packs with mtime 
> in the reversed order as newer objects are always written first.
>
> Let's modify mtime of later pack files (when any) so they appear older 
> than preceding ones when a repack creates multiple packs.

It is a very good companion to f7c22cc (always start looking up objects in
the last used pack first, May 30, 2007) after 10 months ;-)

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> Runtime pack access is done in the pack file mtime order since recent 
> packs are more likely to contain frequently used objects than old packs.
> However the --max-pack-size option can produce multiple packs with mtime 
> in the reversed order as newer objects are always written first.
>
> Let's modify mtime of later pack files (when any) so they appear older 
> than preceding ones when a repack creates multiple packs.
>
> Signed-off-by: Nicolas Pitre <nico@cam.org>
> ---
> diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> index f504cff..4c2ed70 100644
> --- a/builtin-pack-objects.c
> +++ b/builtin-pack-objects.c
> @@ -17,6 +17,8 @@
>  #include "progress.h"
>  #include "refs.h"
>  
> +#include <utime.h>
> +

Hmmm.  Shouldn't this go to git-compat-util.h?

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

[Nicolas Pitre]

On Thu, 13 Mar 2008, Junio C Hamano wrote:

> Nicolas Pitre <nico@cam.org> writes:
> 
> > Runtime pack access is done in the pack file mtime order since recent 
> > packs are more likely to contain frequently used objects than old packs.
> > However the --max-pack-size option can produce multiple packs with mtime 
> > in the reversed order as newer objects are always written first.
> >
> > Let's modify mtime of later pack files (when any) so they appear older 
> > than preceding ones when a repack creates multiple packs.
> >
> > Signed-off-by: Nicolas Pitre <nico@cam.org>
> > ---
> > diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> > index f504cff..4c2ed70 100644
> > --- a/builtin-pack-objects.c
> > +++ b/builtin-pack-objects.c
> > @@ -17,6 +17,8 @@
> >  #include "progress.h"
> >  #include "refs.h"
> >  
> > +#include <utime.h>
> > +
> 
> Hmmm.  Shouldn't this go to git-compat-util.h?

Maybe... if you say so.

The whole header file arrangement logic is somewhat escaping my mind, so 
I simply decided to do the same as in test-chmtime.c.


Nicolas

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

>> > diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
>> > index f504cff..4c2ed70 100644
>> > --- a/builtin-pack-objects.c
>> > +++ b/builtin-pack-objects.c
>> > @@ -17,6 +17,8 @@
>> >  #include "progress.h"
>> >  #include "refs.h"
>> >  
>> > +#include <utime.h>
>> > +
>> 
>> Hmmm.  Shouldn't this go to git-compat-util.h?
>
> Maybe... if you say so.
>
> The whole header file arrangement logic is somewhat escaping my mind, so 
> I simply decided to do the same as in test-chmtime.c.

The general idea is to isolate the place system headers are included at,
so that we have to worry about the inclusion and feature-macro definition
order at only one place (see for example the hack about _ALL_SOURCE to
work around AIX 5.3, yuck).