Unable to index file

Unable to index file

[Ramon Tayag]

Hi everyone,

I've come across a problem that I don't believe lies in Rails.  You
needn't be familiar, I think, with Rails to see what's wrong.

I can't seem to add the files that are in
http://dev.rubyonrails.org/archive/rails_edge.zip

1) Unpack the zip
2) Initialize a git repo inside the folder that was unpacked
3) git add .

See the errors.. :o http://pastie.org/337571

Thanks,
Ramon Tayag

Re: Unable to index file

[Linus Torvalds]

On Fri, 12 Dec 2008, Ramon Tayag wrote:
> 
> I've come across a problem that I don't believe lies in Rails.  You
> needn't be familiar, I think, with Rails to see what's wrong.
> 
> I can't seem to add the files that are in
> http://dev.rubyonrails.org/archive/rails_edge.zip
> 
> 1) Unpack the zip
> 2) Initialize a git repo inside the folder that was unpacked
> 3) git add .
> 
> See the errors.. :o http://pastie.org/337571

What platform/filesystem is this?

Git is rather particular about symlinks, and it looks like your platform 
does something odd, and that makes git unhappy about your symlink.

In particular:

	ls -l vendor/rails/actionpack/test/fixtures/layout_tests/layouts/ 
	...
	lrwxrwxrwx 1 root root 48 2008-12-12 18:22 symlinked -> ../../symlink_parent

notice how the symlink content is "../../symlink_parent", but then take a 
look at the _size_ of the symlink: 48 bytes.

Git expects the lstat() information to match the return from readlink(), 
and it doesn't.

For exact details, see "index_path()" in sha1_file.c:

        case S_IFLNK:   
                len = xsize_t(st->st_size);
                target = xmalloc(len + 1);
                if (readlink(path, target, len + 1) != st->st_size) {
                        char *errstr = strerror(errno);

ie we consider it an error if we get less than st_size characters back 
from readlink().

Now, admittedly git is probably being really annoyingly anal about this 
all, and we probably should loosen the restrictions on it a bit, but I'd 
like to know why it happens. I cannot recall this having been reported 
before, so it's some specific filesystem or OS that causes this, I think.

		Linus

Re: Unable to index file

[Linus Torvalds]

On Fri, 12 Dec 2008, Linus Torvalds wrote:
> 
> Now, admittedly git is probably being really annoyingly anal about this 
> all, and we probably should loosen the restrictions on it a bit, but I'd 
> like to know why it happens. I cannot recall this having been reported 
> before, so it's some specific filesystem or OS that causes this, I think.

Anyway, the "loosen the symlink lstat() requirements" patch would likely 
look something like this. I can't really test it, though, since I only 
have filesystems that have matching lstat()/readlink() sizes.

		Linus
---
 sha1_file.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 0e021c5..222c793 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -2522,9 +2522,9 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object,
 
 int index_path(unsigned char *sha1, const char *path, struct stat *st, int write_object)
 {
-	int fd;
+	int fd, len;
 	char *target;
-	size_t len;
+	size_t bufsize;
 
 	switch (st->st_mode & S_IFMT) {
 	case S_IFREG:
@@ -2537,9 +2537,10 @@ int index_path(unsigned char *sha1, const char *path, struct stat *st, int write
 				     path);
 		break;
 	case S_IFLNK:
-		len = xsize_t(st->st_size);
-		target = xmalloc(len + 1);
-		if (readlink(path, target, len + 1) != st->st_size) {
+		bufsize = 1+xsize_t(st->st_size);
+		target = xmalloc(bufsize);
+		len = readlink(path, target, bufsize);
+		if (len < 0) {
 			char *errstr = strerror(errno);
 			free(target);
 			return error("readlink(\"%s\"): %s", path,

Re: Unable to index file

[Ramon Tayag]

Oh thank you for your very detailed explanation.  I was bothering
people on #rubyonrails and #git but got no answers.

I'm on Ubuntu 8.10.  The files I'm working on live on an NTFS drive
(my "storage" drive; yes, I still have to boot into Windows).  If it
being on NTFS makes a difference I'll try this on ext3 and let you
know what happens :)

Thanks,
Ramon Tayag



On Sat, Dec 13, 2008 at 2:07 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>
> On Fri, 12 Dec 2008, Ramon Tayag wrote:
>>
>> I've come across a problem that I don't believe lies in Rails.  You
>> needn't be familiar, I think, with Rails to see what's wrong.
>>
>> I can't seem to add the files that are in
>> http://dev.rubyonrails.org/archive/rails_edge.zip
>>
>> 1) Unpack the zip
>> 2) Initialize a git repo inside the folder that was unpacked
>> 3) git add .
>>
>> See the errors.. :o http://pastie.org/337571
>
> What platform/filesystem is this?
>
> Git is rather particular about symlinks, and it looks like your platform
> does something odd, and that makes git unhappy about your symlink.
>
> In particular:
>
>        ls -l vendor/rails/actionpack/test/fixtures/layout_tests/layouts/
>        ...
>        lrwxrwxrwx 1 root root 48 2008-12-12 18:22 symlinked -> ../../symlink_parent
>
> notice how the symlink content is "../../symlink_parent", but then take a
> look at the _size_ of the symlink: 48 bytes.
>
> Git expects the lstat() information to match the return from readlink(),
> and it doesn't.
>
> For exact details, see "index_path()" in sha1_file.c:
>
>        case S_IFLNK:
>                len = xsize_t(st->st_size);
>                target = xmalloc(len + 1);
>                if (readlink(path, target, len + 1) != st->st_size) {
>                        char *errstr = strerror(errno);
>
> ie we consider it an error if we get less than st_size characters back
> from readlink().
>
> Now, admittedly git is probably being really annoyingly anal about this
> all, and we probably should loosen the restrictions on it a bit, but I'd
> like to know why it happens. I cannot recall this having been reported
> before, so it's some specific filesystem or OS that causes this, I think.
>
>                Linus
>

Re: Unable to index file

[Linus Torvalds]

On Sat, 13 Dec 2008, Ramon Tayag wrote:
> 
> I'm on Ubuntu 8.10.  The files I'm working on live on an NTFS drive
> (my "storage" drive; yes, I still have to boot into Windows).  If it
> being on NTFS makes a difference I'll try this on ext3 and let you
> know what happens :)

Ok, it's almost certainly the NTFS part.

And at the same time, I'm almost certain that my patch should fix it and 
is the right thing for git to do anyway. 

		Linus