FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016

* FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016
@ 2019-08-15  6:30 ronnie sahlberg
  2019-08-15 15:15 ` Tom Talpey
  2019-08-15 15:48 ` David Disseldorp
  0 siblings, 2 replies; 11+ messages in thread
From: ronnie sahlberg @ 2019-08-15  6:30 UTC (permalink / raw)
  To: linux-cifs

I am seeing issues with how FSCTL_QUERY_ALLOCATED_RANGES behaves under windows,
in particular that it is inconsistent so we can't run certain xfstests
against windows servers.

The behavior can be triggered using the following command from xfstests :
  ./src/seek_sanity_test -s 19 -e 19 /mnt/file
(where /mnt is an smb share mounted from a windows 2016 server.)

In cifs.ko we use this FSCTL to implement both SEEK_HOLE/SEEK_DATA as well as
fiemap(). But since the behavior of this FSCTL is not deteministic it often
cause the tests to fail.

This is a test tool for SEEK_DATA and SEEK_HOLE. As part of this it performs
a check to try to discover if the filesystem supports sparse files or not.
For non-sparse files SEEK_DATA is basically a no-op and SEEK_HOLE just returns
the file size.
Later during the test, the result of whether the file is "sparse" or not
will affect what the expected results are. If this check gets the
sparse-supported wrong the test cases later will fail.

How the check works:
====================
The actual check for whether the file supports being sparse or not is the
following snippet :
        ftruncate(fd, 0);
        bufsz = alloc_size * 2;
        filsz = bufsz * 2;

        buf = do_malloc(bufsz);
        if (!buf)
                goto out;
        memset(buf, 'a', bufsz);

        /* File with 2 allocated blocks.... */
        ret = do_pwrite(fd, buf, bufsz, 0);
        if (ret)
                goto out;

        /* followed by a hole... */
        ret = do_truncate(fd, filsz);
        if (ret)
                goto out;

        /* Is SEEK_DATA and SEEK_HOLE supported in the kernel? */
        pos = lseek(fd, 0, SEEK_HOLE);
        if (pos == -1) {
                fprintf(stderr, "Kernel does not support llseek(2) extension "
                        "SEEK_HOLE. Aborting.\n");
                ret = -1;
                goto out;
        }

        if (pos == filsz) {
                default_behavior = 1;
                fprintf(stderr, "File system supports the default behavior.\n")\
;
        }

I.e.
1, ftruncate to 0
2, write 2M to the start of the file
3, ftruncate the file to be 4Mb
4, SEEK_HOLE and check if it returns 4Mb or no.
If it returns 4Mb then we switch back to default_behavior and we allow
the semantics as if the file is not sparse.
I.e. allow SEEK_DATA to behave as if the file is either sparse or not-sparse.

Also note that if it looks like the sparse-file is not supported then
it prints ""File system supports the default behavior." which may help when
running the test tool.

Strace for this check (when the check failed.)
=============================================
18:22:14.949612 ftruncate(3, 0)         = 0 <0.011513>
18:22:14.963725 mmap(NULL, 2101248, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe8d9f14000 <0.0
00192>
18:22:14.970334 pwrite64(3,
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...,
2097152, 0) = 2097152 <0.002127>
18:22:14.972620 ftruncate(3, 4194304)   = 0 <0.582491>
18:22:15.557308 lseek(3, 0, SEEK_HOLE)  = 4194304 <0.012791>
18:22:15.572457 write(2, "File system supports the default
behavior.\n", 43) = 43 <0.000318>

Example of when the check goes "Wrong". Capture seek_good.cap
=============================================================
The relevant packets here are
1820, set end of file == 0
(1822/1823 update timestamps)
2930, write 2Mb to offset 0
3008, set the file sparse
3010, set end of file to 4M
3019, query-allocated-ranges

In 3019/3020 the server responds that the full 0-4M range is allocated.
This is wrong since file should only be allocated in the first 2Mb at
this stage.
And this makes the test tool think that we don't support sparse files,
and sets default_behavior to 1.

A different run when the check is successful (seek_bad.cap)
============================================================
In the seek_bad capture you can see the same sequence in packets
1869, set end of file == 0
2990, write the first 2M of the file
3067, set file sparse
3069, set end of file to 4M
3078, query-allocated-ranges
But this time, query-allocated-ranges report that only 0-2M is mapped,
which is the correct range,
and thus the test tool assumes that we can handle holes properly.

The captures are ~5MByte each unfiltered so too big for the list.
Email me directly and I will send them to you.

So the question here is what is the actual semantics for sparse files
and query-allocated-ranges on windows?

regards
ronnie sahlberg

^ permalink raw reply	[flat|nested] 11+ messages in thread