query on verify_dump option

query on verify_dump option

[Suresh Dhanarajan]

Hi,

I am trying data integration cases with verify_pattern option.
I am not seeing any dump file generated for the mismatch.
Here are the steps i followed,
Started fio and waited till 30 % writes done the disk and issued a dd
write on the same disc for the first 512 bytes with pattern zero.
then waited till verify gets completed.
I can see the error's in the output log file but not the dump file.

Here is my job file.

[global]
size=10g
direct=1
filename=/dev/sdc
verify_pattern=0x01010101
verify=meta
bs=64k
group_reporting

[write-phase]
rw=write
do_verify=0

[verify-phase]
stonewall
rw=read
do_verify=1
continue_on_error=all
verify_dump=1
~



Let me know if i am missing something over here.

Here is my log file for the verify part,

verify-phase: (groupid=1, jobs=1): err=84 (): pid=9598
  read : io=10240MB, bw=109548KB/s, iops=1711 , runt= 95718msec
    clat (usec): min=443 , max=10326 , avg=487.46, stdev=45.87
     lat (usec): min=443 , max=10326 , avg=487.67, stdev=45.87
    clat percentiles (usec):
     |  1.00th=[  454],  5.00th=[  458], 10.00th=[  462], 20.00th=[  474],
     | 30.00th=[  474], 40.00th=[  478], 50.00th=[  478], 60.00th=[  482],
     | 70.00th=[  486], 80.00th=[  490], 90.00th=[  510], 95.00th=[  532],
     | 99.00th=[  716], 99.50th=[  740], 99.90th=[  764], 99.95th=[  780],
     | 99.99th=[  884]
    bw (KB/s)  : min=105216, max=111360, per=100.00%, avg=109611.56,
stdev=1711.54
    lat (usec) : 500=84.32%, 750=15.45%, 1000=0.23%
    lat (msec) : 2=0.01%, 20=0.01%
  cpu          : usr=16.72%, sys=2.69%, ctx=172103, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=163840/w=0/d=0, short=r=0/w=0/d=0
     errors    : total=1, first_error=84/<Invalid or incomplete
multibyte or wide character>

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=101239KB/s, minb=103669KB/s,
maxb=103669KB/s, mint=103574msec, maxt=103574msec

Run status group 1 (all jobs):
   READ: io=10240MB, aggrb=109548KB/s, minb=112177KB/s,
maxb=112177KB/s, mint=95718msec, maxt=95718msec

Disk stats (read/write):
  sdc: ios=163444/163841, merge=0/0, ticks=75357/92314,
in_queue=167479, util=84.01%
~


2)Also i have one query like verify_pattern option is it possible to
provide known pattern for write and reads this will be very useful in
DI verification test cases

Thanks,
Suresh.D

Re: query on verify_dump option

[Jens Axboe]

On 2012-04-05 17:20, Suresh Dhanarajan wrote:
> Hi,
> 
> I am trying data integration cases with verify_pattern option.
> I am not seeing any dump file generated for the mismatch.
> Here are the steps i followed,
> Started fio and waited till 30 % writes done the disk and issued a dd
> write on the same disc for the first 512 bytes with pattern zero.
> then waited till verify gets completed.
> I can see the error's in the output log file but not the dump file.
> 
> Here is my job file.
> 
> [global]
> size=10g
> direct=1
> filename=/dev/sdc
> verify_pattern=0x01010101
> verify=meta
> bs=64k
> group_reporting
> 
> [write-phase]
> rw=write
> do_verify=0
> 
> [verify-phase]
> stonewall
> rw=read
> do_verify=1
> continue_on_error=all
> verify_dump=1
> ~
> 
> 
> 
> Let me know if i am missing something over here.

You don't get the .received and .expected files?

> 2)Also i have one query like verify_pattern option is it possible to
> provide known pattern for write and reads this will be very useful in
> DI verification test cases

Not sure I understand this one. You are already providing the pattern
you want. What am I missing?

-- 
Jens Axboe

Re: query on verify_dump option

[Suresh Dhanarajan]

Hi,

I am looking for this files in the same location where my fio is and i
dont see the two files that you refereed.

We have option to give pattern during verify only.
What i mentioned is during Write and reads there is no option to
provide the patterns.

thanks,
Suresh.D

On Thu, Apr 5, 2012 at 5:16 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 2012-04-05 17:20, Suresh Dhanarajan wrote:
>> Hi,
>>
>> I am trying data integration cases with verify_pattern option.
>> I am not seeing any dump file generated for the mismatch.
>> Here are the steps i followed,
>> Started fio and waited till 30 % writes done the disk and issued a dd
>> write on the same disc for the first 512 bytes with pattern zero.
>> then waited till verify gets completed.
>> I can see the error's in the output log file but not the dump file.
>>
>> Here is my job file.
>>
>> [global]
>> size=10g
>> direct=1
>> filename=/dev/sdc
>> verify_pattern=0x01010101
>> verify=meta
>> bs=64k
>> group_reporting
>>
>> [write-phase]
>> rw=write
>> do_verify=0
>>
>> [verify-phase]
>> stonewall
>> rw=read
>> do_verify=1
>> continue_on_error=all
>> verify_dump=1
>> ~
>>
>>
>>
>> Let me know if i am missing something over here.
>
> You don't get the .received and .expected files?
>
>> 2)Also i have one query like verify_pattern option is it possible to
>> provide known pattern for write and reads this will be very useful in
>> DI verification test cases
>
> Not sure I understand this one. You are already providing the pattern
> you want. What am I missing?
>
> --
> Jens Axboe
>

Re: query on verify_dump option

[Jens Axboe]

On 2012-04-06 02:28, Suresh Dhanarajan wrote:
> Hi,
> 
> I am looking for this files in the same location where my fio is and i
> dont see the two files that you refereed.

Hmm, you really should, if you have verify_dump=1 set. The pattern
verify is the odd one out, all the other checksum verifiers use shared
code for this. I'll take a look at this and see if I can reproduce it.

> We have option to give pattern during verify only.
> What i mentioned is during Write and reads there is no option to
> provide the patterns.

Ahh, good point. You could just set do_verify=0 for a write workload,
where you define verify_pattern and verify=meta. That should write with
just that pattern. Does that help?

Not sure what you mean for the reads...

-- 
Jens Axboe

Re: query on verify_dump option

[Alireza Haghdoost]

On Wed, Apr 11, 2012 at 3:03 PM, Jens Axboe <axboe@kernel.dk> wrote:
>
> On 2012-04-06 02:28, Suresh Dhanarajan wrote:
> > Hi,
> >
> > I am looking for this files in the same location where my fio is and i
> > dont see the two files that you refereed.
>
> Hmm, you really should, if you have verify_dump=1 set. The pattern
> verify is the odd one out, all the other checksum verifiers use shared
> code for this. I'll take a look at this and see if I can reproduce it.
>

Jens,

Not sure if you had a chance to take a look at this issue. I have just
replicate this issue in my setup. The .received and .excpected files
are missing with verify_dump=1.

Here is my job file, would you please kindly take a look and let me
know if my job file is not correct ?

; writes 512 byte verification blocks until the disk is full,
; then verifies written data
[global]
thread=1
bs=128k
direct=1
iodepth=32
size=10G
ioengine=libaio
verify=crc32c
verify_pattern=0x0123456789abcdfe
verify_interval=512

[write-phase]
filename=/dev/sdd    ; or use a full disk, for example /dev/sda
rw=write
;fill_device=1
do_verify=0
verify_state_save=1

[verify-phase]
stonewall
create_serialize=0
filename=/dev/sdd
rw=read
do_verify=1
verify_only
verify_state_load=1
verify_dump=1
error_dump=1
description=dump_pattern


--Alireza