From: Tom Talpey <ttalpey@microsoft.com>
To: ronnie sahlberg <ronniesahlberg@gmail.com>,
linux-cifs <linux-cifs@vger.kernel.org>
Subject: RE: FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016
Date: Thu, 15 Aug 2019 15:15:01 +0000 [thread overview]
Message-ID: <MWHPR2101MB073136FE90E5C6FCD9F21CA8A0AC0@MWHPR2101MB0731.namprd21.prod.outlook.com> (raw)
In-Reply-To: <CAN05THT0OkbAoNu8ZVSHF-xY7w0ZW4q4i-jTxjNManrnz0OMfg@mail.gmail.com>
> -----Original Message-----
> From: linux-cifs-owner@vger.kernel.org <linux-cifs-owner@vger.kernel.org> On
> Behalf Of ronnie sahlberg
> Sent: Wednesday, August 14, 2019 11:31 PM
> To: linux-cifs <linux-cifs@vger.kernel.org>
> Subject: FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016
>
> I am seeing issues with how FSCTL_QUERY_ALLOCATED_RANGES behaves
> under windows,
> in particular that it is inconsistent so we can't run certain xfstests
> against windows servers.
>
>
> The behavior can be triggered using the following command from xfstests :
> ./src/seek_sanity_test -s 19 -e 19 /mnt/file
> (where /mnt is an smb share mounted from a windows 2016 server.)
>
> In cifs.ko we use this FSCTL to implement both SEEK_HOLE/SEEK_DATA as well
> as
> fiemap(). But since the behavior of this FSCTL is not deteministic it often
> cause the tests to fail.
>
>
> This is a test tool for SEEK_DATA and SEEK_HOLE. As part of this it performs
> a check to try to discover if the filesystem supports sparse files or not.
> For non-sparse files SEEK_DATA is basically a no-op and SEEK_HOLE just returns
> the file size.
> Later during the test, the result of whether the file is "sparse" or not
> will affect what the expected results are. If this check gets the
> sparse-supported wrong the test cases later will fail.
>
>
> How the check works:
> ====================
> The actual check for whether the file supports being sparse or not is the
> following snippet :
> ftruncate(fd, 0);
> bufsz = alloc_size * 2;
> filsz = bufsz * 2;
>
> buf = do_malloc(bufsz);
> if (!buf)
> goto out;
> memset(buf, 'a', bufsz);
>
> /* File with 2 allocated blocks.... */
> ret = do_pwrite(fd, buf, bufsz, 0);
> if (ret)
> goto out;
>
> /* followed by a hole... */
> ret = do_truncate(fd, filsz);
> if (ret)
> goto out;
>
> /* Is SEEK_DATA and SEEK_HOLE supported in the kernel? */
> pos = lseek(fd, 0, SEEK_HOLE);
> if (pos == -1) {
> fprintf(stderr, "Kernel does not support llseek(2) extension "
> "SEEK_HOLE. Aborting.\n");
> ret = -1;
> goto out;
> }
>
> if (pos == filsz) {
> default_behavior = 1;
> fprintf(stderr, "File system supports the default behavior.\n")\
> ;
> }
>
> I.e.
> 1, ftruncate to 0
> 2, write 2M to the start of the file
> 3, ftruncate the file to be 4Mb
> 4, SEEK_HOLE and check if it returns 4Mb or no.
> If it returns 4Mb then we switch back to default_behavior and we allow
> the semantics as if the file is not sparse.
> I.e. allow SEEK_DATA to behave as if the file is either sparse or not-sparse.
>
> Also note that if it looks like the sparse-file is not supported then
> it prints ""File system supports the default behavior." which may help when
> running the test tool.
>
> Strace for this check (when the check failed.)
> =============================================
> 18:22:14.949612 ftruncate(3, 0) = 0 <0.011513>
> 18:22:14.963725 mmap(NULL, 2101248, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe8d9f14000 <0.0
> 00192>
> 18:22:14.970334 pwrite64(3,
> "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...,
> 2097152, 0) = 2097152 <0.002127>
> 18:22:14.972620 ftruncate(3, 4194304) = 0 <0.582491>
> 18:22:15.557308 lseek(3, 0, SEEK_HOLE) = 4194304 <0.012791>
> 18:22:15.572457 write(2, "File system supports the default
> behavior.\n", 43) = 43 <0.000318>
>
>
> Example of when the check goes "Wrong". Capture seek_good.cap
> =============================================================
> The relevant packets here are
> 1820, set end of file == 0
> (1822/1823 update timestamps)
> 2930, write 2Mb to offset 0
> 3008, set the file sparse
> 3010, set end of file to 4M
> 3019, query-allocated-ranges
>
> In 3019/3020 the server responds that the full 0-4M range is allocated.
> This is wrong since file should only be allocated in the first 2Mb at
> this stage.
> And this makes the test tool think that we don't support sparse files,
> and sets default_behavior to 1.
>
> A different run when the check is successful (seek_bad.cap)
> ============================================================
> In the seek_bad capture you can see the same sequence in packets
> 1869, set end of file == 0
> 2990, write the first 2M of the file
> 3067, set file sparse
> 3069, set end of file to 4M
> 3078, query-allocated-ranges
> But this time, query-allocated-ranges report that only 0-2M is mapped,
> which is the correct range,
> and thus the test tool assumes that we can handle holes properly.
>
> The captures are ~5MByte each unfiltered so too big for the list.
> Email me directly and I will send them to you.
>
>
> So the question here is what is the actual semantics for sparse files
> and query-allocated-ranges on windows?
I'll try to get you an answer, but in the meantime just a question...
Does the behavior change if the file is opened with FILE_FLAG_WRITE_THROUGH?
Tom.
next prev parent reply other threads:[~2019-08-15 15:15 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-15 6:30 FSCTL_QUERY_ALLOCATED_RANGES issue with Windows2016 ronnie sahlberg
2019-08-15 15:15 ` Tom Talpey [this message]
2019-08-16 2:17 ` ronnie sahlberg
2019-08-15 15:48 ` David Disseldorp
2019-08-16 2:07 ` ronnie sahlberg
2019-08-16 4:36 ` ronnie sahlberg
2019-08-19 16:31 ` David Disseldorp
2019-08-19 22:03 ` David Disseldorp
2019-08-20 17:47 ` Tom Talpey
2019-08-22 22:46 ` ronnie sahlberg
2019-08-16 9:07 ` David Disseldorp
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=MWHPR2101MB073136FE90E5C6FCD9F21CA8A0AC0@MWHPR2101MB0731.namprd21.prod.outlook.com \
--to=ttalpey@microsoft.com \
--cc=linux-cifs@vger.kernel.org \
--cc=ronniesahlberg@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).