All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Vaughan" <cxt9401@163.com>
To: 'Sitsofe Wheeler' <sitsofe@gmail.com>
Cc: 'fio' <fio@vger.kernel.org>
Subject: 答复: How to ensure split verification will generate the same configs as write phase?
Date: Wed, 28 Feb 2018 16:44:59 +0800	[thread overview]
Message-ID: <000101d3b070$6b165dd0$41431970$@163.com> (raw)
In-Reply-To: <CALjAwxgYcdTe68JPjo6fJHOYc=nD_Tb8Tyd995D1ONbQx3Lvjg@mail.gmail.com>

> ^^^ Here's a problem - you can't randomly write and then verify sequentially via a different run. In your case you need rw=randread.
Yes. It's my fault to paste the original content of the buggy job file. I understand that rw= option should be paired with or without rand- prefix.

> The  randseed is the same across different invocations of fio unless you say otherwise (see randrepeat - http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-randrepeat
> ). So even if you weren't using a verification state file you will re-generate the same pattern if the rest of the job is the same.
Thank you for the explanation. A deep question about the seed comes:) Though randseed is the same across different invocation of fio, is it different for different sections in one job file?

I made another job file as below:
myjob.fio:
	[global] 
	bs=4k    
	direct=1 
	ioengine=sync
	verify=crc32c  
	filename=/dev/sdf
	io_size=10k

	[write-phase]  
	rw=randwrite
	do_verify=0
	write_iolog=iolog

	[verify-phase]
	stonewall
	rw=randread
	do_verify=1

I think these two sections have the same settings except the date direction. So I expect the later one can verify the write-phase. But it failed.	
run it with --debug, and found the offset generated are not the same, so verify abort with error for the first time, as below: 
	random   24382 off rand 259043585
	io       24382 fill_io_u: io_u 0x25ad740: off=71233536/len=4096/ddir=1//dev/sdf
	random   24382 off rand 3179521932
	io       24382 fill_io_u: io_u 0x25ad740: off=874364928/len=4096/ddir=1//dev/sdf
	random   24382 off rand 3621444214
	io       24382 fill_io_u: io_u 0x25ad740: off=995893248/len=4096/ddir=1//dev/sdf
	random   24388 off rand 1094293889
	io       24388 fill_io_u: io_u 0x25ad500: off=300929024/len=4096/ddir=0//dev/sdf
You can see [24382] is write-phase and [24388] is verify-phase, and the first round of them use different seed I suppose.

I search in the source code, and found the following snippet:
	static int setup_random_seeds(struct thread_data *td)
	{
	...
			for (i = 0; i < FIO_RAND_NR_OFFS; i++) {
					td->rand_seeds[i] = seed * td->thread_number + i;
					seed *= 0x9e370001UL;
			}               

Is td->thread_number different for different sections? I thought they both equal to 1 before.

Later, I realize split verify using two sections in one file is not a formal way? Is it better to just write only one write section, and use 'fio --verify-only jobfile' to do the split verification as the updated one below?

myjob.fio updated:
	[global] 
	bs=4k    
	direct=1 
	ioengine=sync
	verify=crc32c  
	filename=/dev/sdf
	io_size=10k

	[write-phase]  
	rw=randwrite
	do_verify=0

Thank you very much again!

Regards,
Vaughan



  reply	other threads:[~2018-02-28  8:45 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-27 10:45 How to ensure split verification will generate the same configs as write phase? Vaughan
2018-02-27 11:59 ` Sitsofe Wheeler
2018-02-28  8:44   ` Vaughan [this message]
2018-02-28 16:15     ` 答复: " Sitsofe Wheeler
2018-03-09  6:11       ` 答复: " Vaughan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='000101d3b070$6b165dd0$41431970$@163.com' \
    --to=cxt9401@163.com \
    --cc=fio@vger.kernel.org \
    --cc=sitsofe@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.