Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Pierre Habouzit]

See below a bug reported against the glibc.  Since the glibc maintainer
dodged that one, I assume the bug indeed is in the documentation of
ftw(3). My manpages are the 3.24-1 Debian package.

IMHO the patch is:

 -fpath is the pathname of the entry relative to dirpath.
 +fpath is the pathname of the entry relative to the current working directory.

POSIX is very vague about what "fpath" should be btw.


On Wed, Mar 31, 2010 at 05:26:09PM -0000, madcoder at debian dot org wrote:
> The man page states:
> 
>        int ftw(const char *dirpath,
>                int (*fn) (const char *fpath, const struct stat *sb,
>                           int typeflag),
>                int nopenfd);
> 
>        [...]
> 
>        fpath is the  pathname  of  the  entry  relative to dirpath.
> 
> Though it appears that it is the pathname of the entry relative to the current 
> working directory.
> 
> FWIW this may be a documentation bug since the similar fts interface clearly 
> states that the similar fts_accpath is relative to the current working 
> directory.
> 
> -- 
>            Summary: ftw doesn't work like documented (may be a documentation
>                     bug)
>            Product: glibc
>            Version: unspecified
>             Status: NEW
>           Severity: normal
>           Priority: P2
>          Component: libc
>         AssignedTo: drepper at redhat dot com
>         ReportedBy: madcoder at debian dot org
>                 CC: glibc-bugs at sources dot redhat dot com
> 
> 
> http://sourceware.org/bugzilla/show_bug.cgi?id=11459
> 
> ------- You are receiving this mail because: -------
> You reported the bug, or are watching the reporter.

On Sat, Apr 03, 2010 at 08:11:46PM -0000, drepper at redhat dot com wrote:
> 
> ------- Additional Comments From drepper at redhat dot com  2010-04-03 20:11 -------
> The man pages do not come with glibc and are in no way authoritative.  You have
> to report this elsewhere.
> 
> -- 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>              Status|NEW                         |RESOLVED
>          Resolution|                            |INVALID
> 
> 
> http://sourceware.org/bugzilla/show_bug.cgi?id=11459
> 
> ------- You are receiving this mail because: -------
> You reported the bug, or are watching the reporter.


-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Michael Kerrisk]

Hello Pierre,

On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> See below a bug reported against the glibc.  Since the glibc maintainer
> dodged that one, I assume the bug indeed is in the documentation of
> ftw(3). My manpages are the 3.24-1 Debian package.

Yes. The man page is clearly incorrect. Thanks for reporting this.

> IMHO the patch is:
>
>  -fpath is the pathname of the entry relative to dirpath.
>  +fpath is the pathname of the entry relative to the current working directory.
>
> POSIX is very vague about what "fpath" should be btw.

(Agreed. It could be more precise.)

I believe the correct text should be this:

       fpath   is  the  pathname  of  the  entry,  and  is
       expressed either as  a  pathname  relative  to  the
       calling  process's current working directory at the
       time of the call to ftw(), if dirpath was expressed
       as a relative pathname, or as an absolute pathname,
       if dirpath was expressed as an  absolute  pathname.

I have updated the man page accordingly, but would welcome
review/checking of this text.

Cheers,

Michael



> On Wed, Mar 31, 2010 at 05:26:09PM -0000, madcoder at debian dot org wrote:
>> The man page states:
>>
>>        int ftw(const char *dirpath,
>>                int (*fn) (const char *fpath, const struct stat *sb,
>>                           int typeflag),
>>                int nopenfd);
>>
>>        [...]
>>
>>        fpath is the  pathname  of  the  entry  relative to dirpath.
>>
>> Though it appears that it is the pathname of the entry relative to the current
>> working directory.
>>
>> FWIW this may be a documentation bug since the similar fts interface clearly
>> states that the similar fts_accpath is relative to the current working
>> directory.
>>
>> --
>>            Summary: ftw doesn't work like documented (may be a documentation
>>                     bug)
>>            Product: glibc
>>            Version: unspecified
>>             Status: NEW
>>           Severity: normal
>>           Priority: P2
>>          Component: libc
>>         AssignedTo: drepper at redhat dot com
>>         ReportedBy: madcoder at debian dot org
>>                 CC: glibc-bugs at sources dot redhat dot com
>>
>>
>> http://sourceware.org/bugzilla/show_bug.cgi?id=11459
>>
>> ------- You are receiving this mail because: -------
>> You reported the bug, or are watching the reporter.
>
> On Sat, Apr 03, 2010 at 08:11:46PM -0000, drepper at redhat dot com wrote:
>>
>> ------- Additional Comments From drepper at redhat dot com  2010-04-03 20:11 -------
>> The man pages do not come with glibc and are in no way authoritative.  You have
>> to report this elsewhere.
>>
>> --
>>            What    |Removed                     |Added
>> ----------------------------------------------------------------------------
>>              Status|NEW                         |RESOLVED
>>          Resolution|                            |INVALID
>>
>>
>> http://sourceware.org/bugzilla/show_bug.cgi?id=11459
>>
>> ------- You are receiving this mail because: -------
>> You reported the bug, or are watching the reporter.
>
>
> --
> ·O·  Pierre Habouzit
> ··O                                                madcoder-8fiUuRrzOP0dnm+yROfE0A@public.gmane.org
> OOO                                                http://www.madism.org
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface" http://blog.man7.org/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Pierre Habouzit]

On Mon, May 24, 2010 at 07:41:49PM +0200, Michael Kerrisk wrote:
> Hello Pierre,
> 
> On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> > See below a bug reported against the glibc.  Since the glibc maintainer
> > dodged that one, I assume the bug indeed is in the documentation of
> > ftw(3). My manpages are the 3.24-1 Debian package.
> 
> Yes. The man page is clearly incorrect. Thanks for reporting this.
> 
> > IMHO the patch is:
> >
> >  -fpath is the pathname of the entry relative to dirpath.
> >  +fpath is the pathname of the entry relative to the current working directory.
> >
> > POSIX is very vague about what "fpath" should be btw.
> 
> (Agreed. It could be more precise.)
> 
> I believe the correct text should be this:
> 
>        fpath   is  the  pathname  of  the  entry,  and  is
>        expressed either as  a  pathname  relative  to  the
>        calling  process's current working directory at the
>        time of the call to ftw(), if dirpath was expressed
>        as a relative pathname, or as an absolute pathname,
>        if dirpath was expressed as an  absolute  pathname.
> 
> I have updated the man page accordingly, but would welcome
> review/checking of this text.

Afaict, it's not correct: ftw may perform chdir() calls, so the pathname
is relative to the current working directory at the time `fn` is called.

I'd rather phrase it that way (minus probable english mistakes):

    fpath is the pathname of the entry, and is either a relative
    pathname to the current working directory of the application when
    `fn` is called, or as an absolute pathname.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Michael Kerrisk]

Hi Pierre,

On Wed, Jun 2, 2010 at 11:42 PM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> On Mon, May 24, 2010 at 07:41:49PM +0200, Michael Kerrisk wrote:
>> Hello Pierre,
>>
>> On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder-zi3NMfd+bKFQFI55V6+gNQ@public.gmane.orgg> wrote:
>> > See below a bug reported against the glibc.  Since the glibc maintainer
>> > dodged that one, I assume the bug indeed is in the documentation of
>> > ftw(3). My manpages are the 3.24-1 Debian package.
>>
>> Yes. The man page is clearly incorrect. Thanks for reporting this.
>>
>> > IMHO the patch is:
>> >
>> >  -fpath is the pathname of the entry relative to dirpath.
>> >  +fpath is the pathname of the entry relative to the current working directory.
>> >
>> > POSIX is very vague about what "fpath" should be btw.
>>
>> (Agreed. It could be more precise.)
>>
>> I believe the correct text should be this:
>>
>>        fpath   is  the  pathname  of  the  entry,  and  is
>>        expressed either as  a  pathname  relative  to  the
>>        calling  process's current working directory at the
>>        time of the call to ftw(), if dirpath was expressed
>>        as a relative pathname, or as an absolute pathname,
>>        if dirpath was expressed as an  absolute  pathname.
>>
>> I have updated the man page accordingly, but would welcome
>> review/checking of this text.
>
> Afaict, it's not correct: ftw may perform chdir() calls, so the pathname
> is relative to the current working directory at the time `fn` is called.
>
> I'd rather phrase it that way (minus probable english mistakes):
>
>    fpath is the pathname of the entry, and is either a relative
>    pathname to the current working directory of the application when
>    `fn` is called, or as an absolute pathname.

Thanks for taking a look at this. However, I *think* your analysis is
wrong, and my proposed changes is right. But, still I'd like some
further confirmation. Please take a look at the the program below, and
the output produced when it runs.

$ cat t.c
#include <stdio.h>
#include <stdlib.h>
#include <ftw.h>

/* Function called by ftw(): print type, i-node number, size in
   bytes, and name of each file. */

static int
displayFileInfo(const char *pathname, const struct stat *sbuf, int type)
{
    printf("%c ", (type == FTW_D) ? 'd' : (type == FTW_DNR) ? 'D' :
                  (type == FTW_F) ? 'f' : (type == FTW_SL) ? 's' :
                  (type == FTW_NS) ? 'N' : '?');
    if (type != FTW_NS)
        printf("%7ld %7lld ", (long) sbuf->st_ino,
                (long long) sbuf->st_size);
    else
        printf("                ");
    printf("%s\n", pathname);

    /* Let's mess with the curent directory during the ftw() call,
       to see what value is passed to 'pathname' in successive calls
       to displayFileInfo() */

    chdir("..");
    system("pwd");

    return 0;           /* to tell ftw() to continue */
}

int
main(int argc, char *argv[])
{
    if (ftw((argc < 2) ? "." : argv[1], displayFileInfo, 10) == -1) {
        perror("ftw");
        exit(EXIT_FAILURE);
    } /* if */

    exit(EXIT_SUCCESS);
}
$ cd dir1
$ find ../dir2
../dir2
../dir2/sub
../dir2/sub/b
../dir2/sub/a
../dir2/bbb
$ ../a.out ../dir2/
d 9863613    4096 ../dir2
/home/mtk/tlpi/dirs_links
d 1236107    4096 ../dir2/sub
/home/mtk/tlpi
f 9176008       0 ../dir2/sub/b
/home/mtk
f 9176007       0 ../dir2/sub/a
/home
f 7635615      38 ../dir2/bbb
/

See how the CWD changes, but the pathname is always expressed with
respect to "dir1"? Do you agree with my text now, or do you still see
some problem?

Thanks,

Michael
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Pierre Habouzit]

On Thu, Jun 03, 2010 at 09:18:22AM +0200, Michael Kerrisk wrote:
> Hi Pierre,
> 
> On Wed, Jun 2, 2010 at 11:42 PM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> > On Mon, May 24, 2010 at 07:41:49PM +0200, Michael Kerrisk wrote:
> >> Hello Pierre,
> >>
> >> On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder@madism.org> wrote:
> >> > See below a bug reported against the glibc.  Since the glibc maintainer
> >> > dodged that one, I assume the bug indeed is in the documentation of
> >> > ftw(3). My manpages are the 3.24-1 Debian package.
> >>
> >> Yes. The man page is clearly incorrect. Thanks for reporting this.
> >>
> >> > IMHO the patch is:
> >> >
> >> >  -fpath is the pathname of the entry relative to dirpath.
> >> >  +fpath is the pathname of the entry relative to the current working directory.
> >> >
> >> > POSIX is very vague about what "fpath" should be btw.
> >>
> >> (Agreed. It could be more precise.)
> >>
> >> I believe the correct text should be this:
> >>
> >>        fpath   is  the  pathname  of  the  entry,  and  is
> >>        expressed either as  a  pathname  relative  to  the
> >>        calling  process's current working directory at the
> >>        time of the call to ftw(), if dirpath was expressed
> >>        as a relative pathname, or as an absolute pathname,
> >>        if dirpath was expressed as an  absolute  pathname.
> >>
> >> I have updated the man page accordingly, but would welcome
> >> review/checking of this text.
> >
> > Afaict, it's not correct: ftw may perform chdir() calls, so the pathname
> > is relative to the current working directory at the time `fn` is called.
> >
> > I'd rather phrase it that way (minus probable english mistakes):
> >
> >    fpath is the pathname of the entry, and is either a relative
> >    pathname to the current working directory of the application when
> >    `fn` is called, or as an absolute pathname.
> 
> Thanks for taking a look at this. However, I *think* your analysis is
> wrong, and my proposed changes is right. But, still I'd like some
> further confirmation. Please take a look at the the program below, and
> the output produced when it runs.

Yeah, that's because you're doing chdir()s during ftw, which is
undefined behaviour as documented in the manpage already IIRC.

The point is, ftw() /may/ decide to do chdir() by itself sometimes, and
then the path is relative to the current working directory as set by
ftw().

I'm pretty sure it's what POSIX authorizes ftw() to perform chdirs.

And when I look at ftw.c in the glibc, it's also pretty much what
happens in the case when you set FTW_CHDIR in the flags: ftw() forces a
chdir before the fn() call, and makes the path relative to this cwd.

It happens that the glibc doesn't seem to perform any kind of chdir() in
the other cases (IOW when FTW_CHDIR isn't set), but I'm pretty sure
POSIX allows ftw() to do so.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Michael Kerrisk]

Hi Pierre,

Thanks for following up on this. I still think the current man page
text is correct. See below.

On Wed, Jun 9, 2010 at 12:46 PM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> On Thu, Jun 03, 2010 at 09:18:22AM +0200, Michael Kerrisk wrote:
>> Hi Pierre,
>>
>> On Wed, Jun 2, 2010 at 11:42 PM, Pierre Habouzit <madcoder-zi3NMfd+bKFQFI55V6+gNQ@public.gmane.orgg> wrote:
>> > On Mon, May 24, 2010 at 07:41:49PM +0200, Michael Kerrisk wrote:
>> >> Hello Pierre,
>> >>
>> >> On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder@madism.org> wrote:
>> >> > See below a bug reported against the glibc.  Since the glibc maintainer
>> >> > dodged that one, I assume the bug indeed is in the documentation of
>> >> > ftw(3). My manpages are the 3.24-1 Debian package.
>> >>
>> >> Yes. The man page is clearly incorrect. Thanks for reporting this.
>> >>
>> >> > IMHO the patch is:
>> >> >
>> >> >  -fpath is the pathname of the entry relative to dirpath.
>> >> >  +fpath is the pathname of the entry relative to the current working directory.
>> >> >
>> >> > POSIX is very vague about what "fpath" should be btw.
>> >>
>> >> (Agreed. It could be more precise.)
>> >>
>> >> I believe the correct text should be this:
>> >>
>> >>        fpath   is  the  pathname  of  the  entry,  and  is
>> >>        expressed either as  a  pathname  relative  to  the
>> >>        calling  process's current working directory at the
>> >>        time of the call to ftw(), if dirpath was expressed
>> >>        as a relative pathname, or as an absolute pathname,
>> >>        if dirpath was expressed as an  absolute  pathname.
>> >>
>> >> I have updated the man page accordingly, but would welcome
>> >> review/checking of this text.
>> >
>> > Afaict, it's not correct: ftw may perform chdir() calls, so the pathname
>> > is relative to the current working directory at the time `fn` is called.
>> >
>> > I'd rather phrase it that way (minus probable english mistakes):
>> >
>> >    fpath is the pathname of the entry, and is either a relative
>> >    pathname to the current working directory of the application when
>> >    `fn` is called, or as an absolute pathname.
>>
>> Thanks for taking a look at this. However, I *think* your analysis is
>> wrong, and my proposed changes is right. But, still I'd like some
>> further confirmation. Please take a look at the the program below, and
>> the output produced when it runs.
>
> Yeah, that's because you're doing chdir()s during ftw, which is
> undefined behaviour as documented in the manpage already IIRC.

True. That is documented in the POSIX page, but not currently in
man-pages. I've fixed that now. Thanks.

> The point is, ftw() /may/ decide to do chdir() by itself sometimes, and
> then the path is relative to the current working directory as set by
> ftw().

I'm not sure why you think it may decide to do this. If this was
unpredictable, then it would create difficulties for the application,
as far as I can tell, since it would take quite some effort to
correctly interpret the pathname supplied to 'fn'. And POSIX seems
fairly clear on the point (at least for nftw()):

       FTW_CHDIR
              If set, nftw() shall change  the  current  working
              directory to each directory as it reports files in
              that directory. If clear, nftw() shall not  change
              the current working directory.

> I'm pretty sure it's what POSIX authorizes ftw() to perform chdirs.

See above.

> And when I look at ftw.c in the glibc, it's also pretty much what
> happens in the case when you set FTW_CHDIR in the flags: ftw() forces a
> chdir before the fn() call, and makes the path relative to this cwd.

Yes to the first part, but as far as I can see, still no to the last
part ("and makes the path relative to this cwd" ). See below.

> It happens that the glibc doesn't seem to perform any kind of chdir() in
> the other cases (IOW when FTW_CHDIR isn't set), but I'm pretty sure
> POSIX allows ftw() to do so.

I don't see anywhere that POSIX authorizes that, and it wouldn't seem
sensible to do so. See above.

Here's a variation on my earlier test program that uses nftw()
instead. If "c" is provided in the second command line argument, it
uses "FTW_CHDIR". Looking at the results, do you agree with my
analysis?

$ cat n.c
#define _GNU_SOURCE
#define _XOPEN_SOURCE 500
#include <ftw.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>


static int
displayFileInfo(const char *fpath, const struct stat *sb,
                int tflag, struct FTW *ftwbuf)
{
    printf("%-3s %2d %7lld   %-40s %d %s\n",
        (tflag == FTW_D) ?   "d"   : (tflag == FTW_DNR) ? "dnr" :
        (tflag == FTW_DP) ?  "dp"  : (tflag == FTW_F) ?   "f" :
        (tflag == FTW_NS) ?  "ns"  : (tflag == FTW_SL) ?  "sl" :
        (tflag == FTW_SLN) ? "sln" : "???",
        ftwbuf->level, (long long) sb->st_size,
        fpath, ftwbuf->base, fpath + ftwbuf->base);
#ifdef DO_CHDIR
    /* Let's mess with the curent directory during the ftw() call,
       to see what value is passed to 'pathname' in successive calls
       to displayFileInfo() */

    chdir("..");
#endif
    system("pwd");
    return 0;           /* To tell nftw() to continue */
}

int
main(int argc, char *argv[])
{
    int flags = 0;

    if (argc > 2 && strchr(argv[2], 'd') != NULL)
        flags |= FTW_DEPTH;
    if (argc > 2 && strchr(argv[2], 'p') != NULL)
        flags |= FTW_PHYS;
    if (argc > 2 && strchr(argv[2], 'c') != NULL)
        flags |= FTW_CHDIR;

    if (nftw((argc < 2) ? "." : argv[1], displayFileInfo, 20, flags) == -1) {
        perror("nftw");
        exit(EXIT_FAILURE);
    }

    exit(EXIT_SUCCESS);
}

$ cc -o n n.c
$ cd dir1
/home/mtk/tlpi/dl/dir1
$ find ../dir2
../dir2
../dir2/sub
../dir2/sub/d
../dir2/sub/b
../dir2/sub/c
../dir2/sub/a
../dir2/bbb
$ ../n ../dir2 c
d    0    4096   ../dir2                                  3 dir2
/home/mtk/tlpi/dirs_links
d    1    4096   ../dir2/sub                              8 sub
/home/mtk/tlpi/dirs_links/dir2
f    2       0   ../dir2/sub/d                            12 d
/home/mtk/tlpi/dirs_links/dir2/sub
f    2       0   ../dir2/sub/b                            12 b
/home/mtk/tlpi/dirs_links/dir2/sub
f    2       0   ../dir2/sub/c                            12 c
/home/mtk/tlpi/dirs_links/dir2/sub
f    2       0   ../dir2/sub/a                            12 a
/home/mtk/tlpi/dirs_links/dir2/sub
f    1      38   ../dir2/bbb                              8 bbb
/home/mtk/tlpi/dirs_links/dir2
$ ../n ../dir2
d    0    4096   ../dir2                                  3 dir2
/home/mtk/tlpi/dl/dir1
d    1    4096   ../dir2/sub                              8 sub
/home/mtk/tlpi/dl/dir1
f    2       0   ../dir2/sub/d                            12 d
/home/mtk/tlpi/dl/dir1
f    2       0   ../dir2/sub/b                            12 b
/home/mtk/tlpi/dl/dir1
f    2       0   ../dir2/sub/c                            12 c
/home/mtk/tlpi/dl/dir1
f    2       0   ../dir2/sub/a                            12 a
/home/mtk/tlpi/dl/dir1
f    1      38   ../dir2/bbb                              8 bbb
/home/mtk/tlpi/dl/dir1

Thanks,

Michael
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Pierre Habouzit]

I think it makes sense indeed.

On Thu, Jun 10, 2010 at 07:04:20AM +0200, Michael Kerrisk wrote:
> Hi Pierre,
> 
> Thanks for following up on this. I still think the current man page
> text is correct. See below.
> 
> On Wed, Jun 9, 2010 at 12:46 PM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> > On Thu, Jun 03, 2010 at 09:18:22AM +0200, Michael Kerrisk wrote:
> >> Hi Pierre,
> >>
> >> On Wed, Jun 2, 2010 at 11:42 PM, Pierre Habouzit <madcoder@madism.org> wrote:
> >> > On Mon, May 24, 2010 at 07:41:49PM +0200, Michael Kerrisk wrote:
> >> >> Hello Pierre,
> >> >>
> >> >> On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder@madism.org> wrote:
> >> >> > See below a bug reported against the glibc.  Since the glibc maintainer
> >> >> > dodged that one, I assume the bug indeed is in the documentation of
> >> >> > ftw(3). My manpages are the 3.24-1 Debian package.
> >> >>
> >> >> Yes. The man page is clearly incorrect. Thanks for reporting this.
> >> >>
> >> >> > IMHO the patch is:
> >> >> >
> >> >> >  -fpath is the pathname of the entry relative to dirpath.
> >> >> >  +fpath is the pathname of the entry relative to the current working directory.
> >> >> >
> >> >> > POSIX is very vague about what "fpath" should be btw.
> >> >>
> >> >> (Agreed. It could be more precise.)
> >> >>
> >> >> I believe the correct text should be this:
> >> >>
> >> >>        fpath   is  the  pathname  of  the  entry,  and  is
> >> >>        expressed either as  a  pathname  relative  to  the
> >> >>        calling  process's current working directory at the
> >> >>        time of the call to ftw(), if dirpath was expressed
> >> >>        as a relative pathname, or as an absolute pathname,
> >> >>        if dirpath was expressed as an  absolute  pathname..
> >> >>
> >> >> I have updated the man page accordingly, but would welcome
> >> >> review/checking of this text.
> >> >
> >> > Afaict, it's not correct: ftw may perform chdir() calls, so the pathname
> >> > is relative to the current working directory at the time `fn` is called.
> >> >
> >> > I'd rather phrase it that way (minus probable english mistakes):
> >> >
> >> >    fpath is the pathname of the entry, and is either a relative
> >> >    pathname to the current working directory of the application when
> >> >    `fn` is called, or as an absolute pathname.
> >>
> >> Thanks for taking a look at this. However, I *think* your analysis is
> >> wrong, and my proposed changes is right. But, still I'd like some
> >> further confirmation. Please take a look at the the program below, and
> >> the output produced when it runs.
> >
> > Yeah, that's because you're doing chdir()s during ftw, which is
> > undefined behaviour as documented in the manpage already IIRC.
> 
> True. That is documented in the POSIX page, but not currently in
> man-pages. I've fixed that now. Thanks.
> 
> > The point is, ftw() /may/ decide to do chdir() by itself sometimes, and
> > then the path is relative to the current working directory as set by
> > ftw().
> 
> I'm not sure why you think it may decide to do this. If this was
> unpredictable, then it would create difficulties for the application,
> as far as I can tell, since it would take quite some effort to
> correctly interpret the pathname supplied to 'fn'. And POSIX seems
> fairly clear on the point (at least for nftw()):
> 
>        FTW_CHDIR
>               If set, nftw() shall change  the  current  working
>               directory to each directory as it reports files in
>               that directory. If clear, nftw() shall not  change
>               the current working directory.
> 
> > I'm pretty sure it's what POSIX authorizes ftw() to perform chdirs.
> 
> See above.
> 
> > And when I look at ftw.c in the glibc, it's also pretty much what
> > happens in the case when you set FTW_CHDIR in the flags: ftw() forces a
> > chdir before the fn() call, and makes the path relative to this cwd.
> 
> Yes to the first part, but as far as I can see, still no to the last
> part ("and makes the path relative to this cwd" ). See below.
> 
> > It happens that the glibc doesn't seem to perform any kind of chdir() in
> > the other cases (IOW when FTW_CHDIR isn't set), but I'm pretty sure
> > POSIX allows ftw() to do so.
> 
> I don't see anywhere that POSIX authorizes that, and it wouldn't seem
> sensible to do so. See above.
> 
> Here's a variation on my earlier test program that uses nftw()
> instead. If "c" is provided in the second command line argument, it
> uses "FTW_CHDIR". Looking at the results, do you agree with my
> analysis?
> 
> $ cat n.c
> #define _GNU_SOURCE
> #define _XOPEN_SOURCE 500
> #include <ftw.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> 
> 
> static int
> displayFileInfo(const char *fpath, const struct stat *sb,
>                 int tflag, struct FTW *ftwbuf)
> {
>     printf("%-3s %2d %7lld   %-40s %d %s\n",
>         (tflag == FTW_D) ?   "d"   : (tflag == FTW_DNR) ? "dnr" :
>         (tflag == FTW_DP) ?  "dp"  : (tflag == FTW_F) ?   "f" :
>         (tflag == FTW_NS) ?  "ns"  : (tflag == FTW_SL) ?  "sl" :
>         (tflag == FTW_SLN) ? "sln" : "???",
>         ftwbuf->level, (long long) sb->st_size,
>         fpath, ftwbuf->base, fpath + ftwbuf->base);
> #ifdef DO_CHDIR
>     /* Let's mess with the curent directory during the ftw() call,
>        to see what value is passed to 'pathname' in successive calls
>        to displayFileInfo() */
> 
>     chdir("..");
> #endif
>     system("pwd");
>     return 0;           /* To tell nftw() to continue */
> }
> 
> int
> main(int argc, char *argv[])
> {
>     int flags = 0;
> 
>     if (argc > 2 && strchr(argv[2], 'd') != NULL)
>         flags |= FTW_DEPTH;
>     if (argc > 2 && strchr(argv[2], 'p') != NULL)
>         flags |= FTW_PHYS;
>     if (argc > 2 && strchr(argv[2], 'c') != NULL)
>         flags |= FTW_CHDIR;
> 
>     if (nftw((argc < 2) ? "." : argv[1], displayFileInfo, 20, flags) == -1) {
>         perror("nftw");
>         exit(EXIT_FAILURE);
>     }
> 
>     exit(EXIT_SUCCESS);
> }
> 
> $ cc -o n n.c
> $ cd dir1
> /home/mtk/tlpi/dl/dir1
> $ find ../dir2
> .../dir2
> .../dir2/sub
> .../dir2/sub/d
> .../dir2/sub/b
> .../dir2/sub/c
> .../dir2/sub/a
> .../dir2/bbb
> $ ../n ../dir2 c
> d    0    4096   ../dir2                                  3 dir2
> /home/mtk/tlpi/dirs_links
> d    1    4096   ../dir2/sub                              8 sub
> /home/mtk/tlpi/dirs_links/dir2
> f    2       0   ../dir2/sub/d                            12 d
> /home/mtk/tlpi/dirs_links/dir2/sub
> f    2       0   ../dir2/sub/b                            12 b
> /home/mtk/tlpi/dirs_links/dir2/sub
> f    2       0   ../dir2/sub/c                            12 c
> /home/mtk/tlpi/dirs_links/dir2/sub
> f    2       0   ../dir2/sub/a                            12 a
> /home/mtk/tlpi/dirs_links/dir2/sub
> f    1      38   ../dir2/bbb                              8 bbb
> /home/mtk/tlpi/dirs_links/dir2
> $ ../n ../dir2
> d    0    4096   ../dir2                                  3 dir2
> /home/mtk/tlpi/dl/dir1
> d    1    4096   ../dir2/sub                              8 sub
> /home/mtk/tlpi/dl/dir1
> f    2       0   ../dir2/sub/d                            12 d
> /home/mtk/tlpi/dl/dir1
> f    2       0   ../dir2/sub/b                            12 b
> /home/mtk/tlpi/dl/dir1
> f    2       0   ../dir2/sub/c                            12 c
> /home/mtk/tlpi/dl/dir1
> f    2       0   ../dir2/sub/a                            12 a
> /home/mtk/tlpi/dl/dir1
> f    1      38   ../dir2/bbb                              8 bbb
> /home/mtk/tlpi/dl/dir1
> 
> Thanks,
> 
> Michael

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug libc/11459] New: ftw doesn't work like documented (may be a documentation bug)

[Michael Kerrisk]

On Thu, Jun 24, 2010 at 10:12 PM, Pierre Habouzit <madcoder-zi3NMfd+bKEdnm+yROfE0A@public.gmane.org> wrote:
> I think it makes sense indeed.

Thanks Pierre. It's always good to have someone else confirm.

Cheers,

Michael


> On Thu, Jun 10, 2010 at 07:04:20AM +0200, Michael Kerrisk wrote:
>> Hi Pierre,
>>
>> Thanks for following up on this. I still think the current man page
>> text is correct. See below.
>>
>> On Wed, Jun 9, 2010 at 12:46 PM, Pierre Habouzit <madcoder-zi3NMfd+bKFQFI55V6+gNQ@public.gmane.orgg> wrote:
>> > On Thu, Jun 03, 2010 at 09:18:22AM +0200, Michael Kerrisk wrote:
>> >> Hi Pierre,
>> >>
>> >> On Wed, Jun 2, 2010 at 11:42 PM, Pierre Habouzit <madcoder@madism.org> wrote:
>> >> > On Mon, May 24, 2010 at 07:41:49PM +0200, Michael Kerrisk wrote:
>> >> >> Hello Pierre,
>> >> >>
>> >> >> On Tue, Apr 6, 2010 at 11:03 AM, Pierre Habouzit <madcoder@madism.org> wrote:
>> >> >> > See below a bug reported against the glibc.  Since the glibc maintainer
>> >> >> > dodged that one, I assume the bug indeed is in the documentation of
>> >> >> > ftw(3). My manpages are the 3.24-1 Debian package.
>> >> >>
>> >> >> Yes. The man page is clearly incorrect. Thanks for reporting this.
>> >> >>
>> >> >> > IMHO the patch is:
>> >> >> >
>> >> >> >  -fpath is the pathname of the entry relative to dirpath.
>> >> >> >  +fpath is the pathname of the entry relative to the current working directory.
>> >> >> >
>> >> >> > POSIX is very vague about what "fpath" should be btw.
>> >> >>
>> >> >> (Agreed. It could be more precise.)
>> >> >>
>> >> >> I believe the correct text should be this:
>> >> >>
>> >> >>        fpath   is  the  pathname  of  the  entry,  and  is
>> >> >>        expressed either as  a  pathname  relative  to  the
>> >> >>        calling  process's current working directory at the
>> >> >>        time of the call to ftw(), if dirpath was expressed
>> >> >>        as a relative pathname, or as an absolute pathname,
>> >> >>        if dirpath was expressed as an  absolute  pathname..
>> >> >>
>> >> >> I have updated the man page accordingly, but would welcome
>> >> >> review/checking of this text.
>> >> >
>> >> > Afaict, it's not correct: ftw may perform chdir() calls, so the pathname
>> >> > is relative to the current working directory at the time `fn` is called.
>> >> >
>> >> > I'd rather phrase it that way (minus probable english mistakes):
>> >> >
>> >> >    fpath is the pathname of the entry, and is either a relative
>> >> >    pathname to the current working directory of the application when
>> >> >    `fn` is called, or as an absolute pathname.
>> >>
>> >> Thanks for taking a look at this. However, I *think* your analysis is
>> >> wrong, and my proposed changes is right. But, still I'd like some
>> >> further confirmation. Please take a look at the the program below, and
>> >> the output produced when it runs.
>> >
>> > Yeah, that's because you're doing chdir()s during ftw, which is
>> > undefined behaviour as documented in the manpage already IIRC.
>>
>> True. That is documented in the POSIX page, but not currently in
>> man-pages. I've fixed that now. Thanks.
>>
>> > The point is, ftw() /may/ decide to do chdir() by itself sometimes, and
>> > then the path is relative to the current working directory as set by
>> > ftw().
>>
>> I'm not sure why you think it may decide to do this. If this was
>> unpredictable, then it would create difficulties for the application,
>> as far as I can tell, since it would take quite some effort to
>> correctly interpret the pathname supplied to 'fn'. And POSIX seems
>> fairly clear on the point (at least for nftw()):
>>
>>        FTW_CHDIR
>>               If set, nftw() shall change  the  current  working
>>               directory to each directory as it reports files in
>>               that directory. If clear, nftw() shall not  change
>>               the current working directory.
>>
>> > I'm pretty sure it's what POSIX authorizes ftw() to perform chdirs.
>>
>> See above.
>>
>> > And when I look at ftw.c in the glibc, it's also pretty much what
>> > happens in the case when you set FTW_CHDIR in the flags: ftw() forces a
>> > chdir before the fn() call, and makes the path relative to this cwd.
>>
>> Yes to the first part, but as far as I can see, still no to the last
>> part ("and makes the path relative to this cwd" ). See below.
>>
>> > It happens that the glibc doesn't seem to perform any kind of chdir() in
>> > the other cases (IOW when FTW_CHDIR isn't set), but I'm pretty sure
>> > POSIX allows ftw() to do so.
>>
>> I don't see anywhere that POSIX authorizes that, and it wouldn't seem
>> sensible to do so. See above.
>>
>> Here's a variation on my earlier test program that uses nftw()
>> instead. If "c" is provided in the second command line argument, it
>> uses "FTW_CHDIR". Looking at the results, do you agree with my
>> analysis?
>>
>> $ cat n.c
>> #define _GNU_SOURCE
>> #define _XOPEN_SOURCE 500
>> #include <ftw.h>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <string.h>
>>
>>
>> static int
>> displayFileInfo(const char *fpath, const struct stat *sb,
>>                 int tflag, struct FTW *ftwbuf)
>> {
>>     printf("%-3s %2d %7lld   %-40s %d %s\n",
>>         (tflag == FTW_D) ?   "d"   : (tflag == FTW_DNR) ? "dnr" :
>>         (tflag == FTW_DP) ?  "dp"  : (tflag == FTW_F) ?   "f" :
>>         (tflag == FTW_NS) ?  "ns"  : (tflag == FTW_SL) ?  "sl" :
>>         (tflag == FTW_SLN) ? "sln" : "???",
>>         ftwbuf->level, (long long) sb->st_size,
>>         fpath, ftwbuf->base, fpath + ftwbuf->base);
>> #ifdef DO_CHDIR
>>     /* Let's mess with the curent directory during the ftw() call,
>>        to see what value is passed to 'pathname' in successive calls
>>        to displayFileInfo() */
>>
>>     chdir("..");
>> #endif
>>     system("pwd");
>>     return 0;           /* To tell nftw() to continue */
>> }
>>
>> int
>> main(int argc, char *argv[])
>> {
>>     int flags = 0;
>>
>>     if (argc > 2 && strchr(argv[2], 'd') != NULL)
>>         flags |= FTW_DEPTH;
>>     if (argc > 2 && strchr(argv[2], 'p') != NULL)
>>         flags |= FTW_PHYS;
>>     if (argc > 2 && strchr(argv[2], 'c') != NULL)
>>         flags |= FTW_CHDIR;
>>
>>     if (nftw((argc < 2) ? "." : argv[1], displayFileInfo, 20, flags) == -1) {
>>         perror("nftw");
>>         exit(EXIT_FAILURE);
>>     }
>>
>>     exit(EXIT_SUCCESS);
>> }
>>
>> $ cc -o n n.c
>> $ cd dir1
>> /home/mtk/tlpi/dl/dir1
>> $ find ../dir2
>> .../dir2
>> .../dir2/sub
>> .../dir2/sub/d
>> .../dir2/sub/b
>> .../dir2/sub/c
>> .../dir2/sub/a
>> .../dir2/bbb
>> $ ../n ../dir2 c
>> d    0    4096   ../dir2                                  3 dir2
>> /home/mtk/tlpi/dirs_links
>> d    1    4096   ../dir2/sub                              8 sub
>> /home/mtk/tlpi/dirs_links/dir2
>> f    2       0   ../dir2/sub/d                            12 d
>> /home/mtk/tlpi/dirs_links/dir2/sub
>> f    2       0   ../dir2/sub/b                            12 b
>> /home/mtk/tlpi/dirs_links/dir2/sub
>> f    2       0   ../dir2/sub/c                            12 c
>> /home/mtk/tlpi/dirs_links/dir2/sub
>> f    2       0   ../dir2/sub/a                            12 a
>> /home/mtk/tlpi/dirs_links/dir2/sub
>> f    1      38   ../dir2/bbb                              8 bbb
>> /home/mtk/tlpi/dirs_links/dir2
>> $ ../n ../dir2
>> d    0    4096   ../dir2                                  3 dir2
>> /home/mtk/tlpi/dl/dir1
>> d    1    4096   ../dir2/sub                              8 sub
>> /home/mtk/tlpi/dl/dir1
>> f    2       0   ../dir2/sub/d                            12 d
>> /home/mtk/tlpi/dl/dir1
>> f    2       0   ../dir2/sub/b                            12 b
>> /home/mtk/tlpi/dl/dir1
>> f    2       0   ../dir2/sub/c                            12 c
>> /home/mtk/tlpi/dl/dir1
>> f    2       0   ../dir2/sub/a                            12 a
>> /home/mtk/tlpi/dl/dir1
>> f    1      38   ../dir2/bbb                              8 bbb
>> /home/mtk/tlpi/dl/dir1
>>
>> Thanks,
>>
>> Michael
>
> --
> ·O·  Pierre Habouzit
> ··O                                                madcoder-8fiUuRrzOP0dnm+yROfE0A@public.gmane.org
> OOO                                                http://www.madism.org
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface" http://blog.man7.org/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html