3.10: discard/trim support on md-raid1?

3.10: discard/trim support on md-raid1?

[Justin Piszcz]

Hello,

Running 3.10 and I see the following for an md-raid1 of two SSDs:

Checking /sys/block/md1/queue:
add_random: 0
discard_granularity: 512
discard_max_bytes: 2147450880
discard_zeroes_data: 0
hw_sector_size: 512
iostats: 0
logical_block_size: 512
max_hw_sectors_kb: 32767
max_integrity_segments: 0
max_sectors_kb: 512
max_segment_size: 65536
max_segments: 168
minimum_io_size: 512
nomerges: 0
nr_requests: 128
optimal_io_size: 0
physical_block_size: 512
read_ahead_kb: 8192
rotational: 1
rq_affinity: 0
scheduler: none
write_same_max_bytes: 0

What should be seen:
rotational: 0
And possibly:
discard_zeroes_data: 1

Can anyone confirm if there is a workaround to allow TRIM when using
md-raid1?

Some related discussion here:
http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
ml
http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html


Justin.

3.10: discard/trim support on md-raid1?

[Justin Piszcz]

Hello,

Running 3.10 and I see the following for an md-raid1 of two SSDs:

Checking /sys/block/md1/queue:
add_random: 0
discard_granularity: 512
discard_max_bytes: 2147450880
discard_zeroes_data: 0
hw_sector_size: 512
iostats: 0
logical_block_size: 512
max_hw_sectors_kb: 32767
max_integrity_segments: 0
max_sectors_kb: 512
max_segment_size: 65536
max_segments: 168
minimum_io_size: 512
nomerges: 0
nr_requests: 128
optimal_io_size: 0
physical_block_size: 512
read_ahead_kb: 8192
rotational: 1
rq_affinity: 0
scheduler: none
write_same_max_bytes: 0

What should be seen:
rotational: 0
And possibly:
discard_zeroes_data: 1

Can anyone confirm if there is a workaround to allow TRIM when using
md-raid1?

Some related discussion here:
http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
ml
http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html


Justin.

Re: 3.10: discard/trim support on md-raid1?

[Brad Campbell]

On 13/07/13 18:34, Justin Piszcz wrote:
> And possibly:
> discard_zeroes_data: 1

Does it though?

Here's my 6 x SSD RAID10 that definitely discards.

brad@srv:~$ grep . /sys/block/md2/queue/*
/sys/block/md2/queue/add_random:0
/sys/block/md2/queue/discard_granularity:33553920
/sys/block/md2/queue/discard_max_bytes:65536
/sys/block/md2/queue/discard_zeroes_data:0
/sys/block/md2/queue/hw_sector_size:512
/sys/block/md2/queue/iostats:0
/sys/block/md2/queue/logical_block_size:512
/sys/block/md2/queue/max_hw_sectors_kb:16383
/sys/block/md2/queue/max_integrity_segments:0
/sys/block/md2/queue/max_sectors_kb:512
/sys/block/md2/queue/max_segments:128
/sys/block/md2/queue/max_segment_size:65536
/sys/block/md2/queue/minimum_io_size:65536
/sys/block/md2/queue/nomerges:0
/sys/block/md2/queue/nr_requests:128
/sys/block/md2/queue/optimal_io_size:196608
/sys/block/md2/queue/physical_block_size:512
/sys/block/md2/queue/read_ahead_kb:384
/sys/block/md2/queue/rotational:1
/sys/block/md2/queue/rq_affinity:0
/sys/block/md2/queue/scheduler:none
/sys/block/md2/queue/write_same_max_bytes:0

Mine does not have discard_zeroes_data == 1 which is correct as 3 of the drives are Samsung 830's 
and they don't return deterministic after trim.

I can verify that discard gets passed down the stack as I get a non-zero mismatch count after 
running a trim on the filesystem. Checking the mirrors manually sees 0 returned from the Intel 
330's, and apparently random data returned from the 830's.

Regards,
Brad

Re: 3.10: discard/trim support on md-raid1?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

On Sat, 13 Jul 2013 06:34:19 -0400 "Justin Piszcz" <jpiszcz@lucidpixels.com>
wrote:

> Hello,
> 
> Running 3.10 and I see the following for an md-raid1 of two SSDs:
> 
> Checking /sys/block/md1/queue:
> add_random: 0
> discard_granularity: 512
> discard_max_bytes: 2147450880
> discard_zeroes_data: 0
> hw_sector_size: 512
> iostats: 0
> logical_block_size: 512
> max_hw_sectors_kb: 32767
> max_integrity_segments: 0
> max_sectors_kb: 512
> max_segment_size: 65536
> max_segments: 168
> minimum_io_size: 512
> nomerges: 0
> nr_requests: 128
> optimal_io_size: 0
> physical_block_size: 512
> read_ahead_kb: 8192
> rotational: 1
> rq_affinity: 0
> scheduler: none
> write_same_max_bytes: 0
> 
> What should be seen:
> rotational: 0

What has "rotational" got to do with "supports discard"?
There may be some correlation, but it isn't causal.

> And possibly:
> discard_zeroes_data: 1

This should be set as the 'or' of the same value from component devices.  And
does not enable or disable the use of discard.

I don't think that "does this device support discard" appears in sysfs.

I believe trim does work on md/raid1 if the underlying devices all support it.

NeilBrown



> 
> Can anyone confirm if there is a workaround to allow TRIM when using
> md-raid1?
> 
> Some related discussion here:
> http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
> ml
> http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html
> 
> 
> Justin.
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: 3.10: discard/trim support on md-raid1?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

On Sat, 13 Jul 2013 06:34:19 -0400 "Justin Piszcz" <jpiszcz@lucidpixels.com>
wrote:

> Hello,
> 
> Running 3.10 and I see the following for an md-raid1 of two SSDs:
> 
> Checking /sys/block/md1/queue:
> add_random: 0
> discard_granularity: 512
> discard_max_bytes: 2147450880
> discard_zeroes_data: 0
> hw_sector_size: 512
> iostats: 0
> logical_block_size: 512
> max_hw_sectors_kb: 32767
> max_integrity_segments: 0
> max_sectors_kb: 512
> max_segment_size: 65536
> max_segments: 168
> minimum_io_size: 512
> nomerges: 0
> nr_requests: 128
> optimal_io_size: 0
> physical_block_size: 512
> read_ahead_kb: 8192
> rotational: 1
> rq_affinity: 0
> scheduler: none
> write_same_max_bytes: 0
> 
> What should be seen:
> rotational: 0

What has "rotational" got to do with "supports discard"?
There may be some correlation, but it isn't causal.

> And possibly:
> discard_zeroes_data: 1

This should be set as the 'or' of the same value from component devices.  And
does not enable or disable the use of discard.

I don't think that "does this device support discard" appears in sysfs.

I believe trim does work on md/raid1 if the underlying devices all support it.

NeilBrown



> 
> Can anyone confirm if there is a workaround to allow TRIM when using
> md-raid1?
> 
> Some related discussion here:
> http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
> ml
> http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html
> 
> 
> Justin.
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: 3.10: discard/trim support on md-raid1?

[Justin Piszcz]

Thanks for the replies,

After some further testing..
When I ran a repair on the md's sync_action, the system would reduce
I/O to the RAID-1 to 14kb/s or even less when it hit a certain number
of blocks and effectively locked the system every time.
It turned out to be a bad SSD (it also failed Intel's Secure Erase), I RMA'd it.
Interesting though that it did not drop out of the array but froze the
system (the failure scenario was odd).

Justin.



On Tue, Jul 16, 2013 at 3:15 AM, NeilBrown <neilb@suse.de> wrote:
> On Sat, 13 Jul 2013 06:34:19 -0400 "Justin Piszcz" <jpiszcz@lucidpixels.com>
> wrote:
>
>> Hello,
>>
>> Running 3.10 and I see the following for an md-raid1 of two SSDs:
>>
>> Checking /sys/block/md1/queue:
>> add_random: 0
>> discard_granularity: 512
>> discard_max_bytes: 2147450880
>> discard_zeroes_data: 0
>> hw_sector_size: 512
>> iostats: 0
>> logical_block_size: 512
>> max_hw_sectors_kb: 32767
>> max_integrity_segments: 0
>> max_sectors_kb: 512
>> max_segment_size: 65536
>> max_segments: 168
>> minimum_io_size: 512
>> nomerges: 0
>> nr_requests: 128
>> optimal_io_size: 0
>> physical_block_size: 512
>> read_ahead_kb: 8192
>> rotational: 1
>> rq_affinity: 0
>> scheduler: none
>> write_same_max_bytes: 0
>>
>> What should be seen:
>> rotational: 0
>
> What has "rotational" got to do with "supports discard"?
> There may be some correlation, but it isn't causal.
>
>> And possibly:
>> discard_zeroes_data: 1
>
> This should be set as the 'or' of the same value from component devices.  And
> does not enable or disable the use of discard.
>
> I don't think that "does this device support discard" appears in sysfs.
>
> I believe trim does work on md/raid1 if the underlying devices all support it.
>
> NeilBrown
>
>
>
>>
>> Can anyone confirm if there is a workaround to allow TRIM when using
>> md-raid1?
>>
>> Some related discussion here:
>> http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
>> ml
>> http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html
>>
>>
>> Justin.
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: 3.10: discard/trim support on md-raid1?

[David Lang]

Part of this depends on the exact failure mode. I've seen cases where drives 
fail, and the drive does a bunch of retries, then the OS does a bunch of 
retries, and eventually the read fails, but in the meantime, everything stalls 
for a long time.

I've even seen the same thing in at least one case where there was a hardware 
RAID card in use.

David Lang

On Tue, 16 Jul 2013, Justin Piszcz wrote:

> Thanks for the replies,
>
> After some further testing..
> When I ran a repair on the md's sync_action, the system would reduce
> I/O to the RAID-1 to 14kb/s or even less when it hit a certain number
> of blocks and effectively locked the system every time.
> It turned out to be a bad SSD (it also failed Intel's Secure Erase), I RMA'd it.
> Interesting though that it did not drop out of the array but froze the
> system (the failure scenario was odd).
>
> Justin.
>
>
>
> On Tue, Jul 16, 2013 at 3:15 AM, NeilBrown <neilb@suse.de> wrote:
>> On Sat, 13 Jul 2013 06:34:19 -0400 "Justin Piszcz" <jpiszcz@lucidpixels.com>
>> wrote:
>>
>>> Hello,
>>>
>>> Running 3.10 and I see the following for an md-raid1 of two SSDs:
>>>
>>> Checking /sys/block/md1/queue:
>>> add_random: 0
>>> discard_granularity: 512
>>> discard_max_bytes: 2147450880
>>> discard_zeroes_data: 0
>>> hw_sector_size: 512
>>> iostats: 0
>>> logical_block_size: 512
>>> max_hw_sectors_kb: 32767
>>> max_integrity_segments: 0
>>> max_sectors_kb: 512
>>> max_segment_size: 65536
>>> max_segments: 168
>>> minimum_io_size: 512
>>> nomerges: 0
>>> nr_requests: 128
>>> optimal_io_size: 0
>>> physical_block_size: 512
>>> read_ahead_kb: 8192
>>> rotational: 1
>>> rq_affinity: 0
>>> scheduler: none
>>> write_same_max_bytes: 0
>>>
>>> What should be seen:
>>> rotational: 0
>>
>> What has "rotational" got to do with "supports discard"?
>> There may be some correlation, but it isn't causal.
>>
>>> And possibly:
>>> discard_zeroes_data: 1
>>
>> This should be set as the 'or' of the same value from component devices.  And
>> does not enable or disable the use of discard.
>>
>> I don't think that "does this device support discard" appears in sysfs.
>>
>> I believe trim does work on md/raid1 if the underlying devices all support it.
>>
>> NeilBrown
>>
>>
>>
>>>
>>> Can anyone confirm if there is a workaround to allow TRIM when using
>>> md-raid1?
>>>
>>> Some related discussion here:
>>> http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
>>> ml
>>> http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html
>>>
>>>
>>> Justin.
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: 3.10: discard/trim support on md-raid1?

[CoolCold]

Neil, I've tryed to look around commit logs but failed to find commit
where discard/trim were added.
I was looking via
http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/log/drivers/md?id=9f2a940965286754f3a34d5737c3097c05db8725&qt=grep&q=discard+support
, tryed just "discard" without support - found commits about dm, not
md, though.

Can you shed some light here? Thanks in advance.

(resent in plaintext mode, please reply to this one)


On Tue, Jul 16, 2013 at 11:15 AM, NeilBrown <neilb@suse.de> wrote:
> On Sat, 13 Jul 2013 06:34:19 -0400 "Justin Piszcz" <jpiszcz@lucidpixels.com>
> wrote:
>
>> Hello,
>>
>> Running 3.10 and I see the following for an md-raid1 of two SSDs:
>>
>> Checking /sys/block/md1/queue:
>> add_random: 0
>> discard_granularity: 512
>> discard_max_bytes: 2147450880
>> discard_zeroes_data: 0
>> hw_sector_size: 512
>> iostats: 0
>> logical_block_size: 512
>> max_hw_sectors_kb: 32767
>> max_integrity_segments: 0
>> max_sectors_kb: 512
>> max_segment_size: 65536
>> max_segments: 168
>> minimum_io_size: 512
>> nomerges: 0
>> nr_requests: 128
>> optimal_io_size: 0
>> physical_block_size: 512
>> read_ahead_kb: 8192
>> rotational: 1
>> rq_affinity: 0
>> scheduler: none
>> write_same_max_bytes: 0
>>
>> What should be seen:
>> rotational: 0
>
> What has "rotational" got to do with "supports discard"?
> There may be some correlation, but it isn't causal.
>
>> And possibly:
>> discard_zeroes_data: 1
>
> This should be set as the 'or' of the same value from component devices.  And
> does not enable or disable the use of discard.
>
> I don't think that "does this device support discard" appears in sysfs.
>
> I believe trim does work on md/raid1 if the underlying devices all support it.
>
> NeilBrown
>
>
>
>>
>> Can anyone confirm if there is a workaround to allow TRIM when using
>> md-raid1?
>>
>> Some related discussion here:
>> http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
>> ml
>> http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html
>>
>>
>> Justin.
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Best regards,
[COOLCOLD-RIPN]

Re: 3.10: discard/trim support on md-raid1?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2913 bytes --]

On Wed, 17 Jul 2013 00:44:15 +0400 CoolCold <coolthecold@gmail.com> wrote:

> Neil, I've tryed to look around commit logs but failed to find commit
> where discard/trim were added.
> I was looking via
> http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/log/drivers/md?id=9f2a940965286754f3a34d5737c3097c05db8725&qt=grep&q=discard+support
> , tryed just "discard" without support - found commits about dm, not
> md, though.
> 
> Can you shed some light here? Thanks in advance.
> 
> (resent in plaintext mode, please reply to this one)

Did you try:
    grep -i discard drivers/md/raid1.c
??

commit 2ff8cc2c6d4e323de71a42affeb3041fa17d5b10
Author: Shaohua Li <shli@fusionio.com>
Date:   Thu Oct 11 13:28:54 2012 +1100

    md: raid 1 supports TRIM

is the commit that introduced discard support for md/raid1.c

NeilBrown


> 
> 
> On Tue, Jul 16, 2013 at 11:15 AM, NeilBrown <neilb@suse.de> wrote:
> > On Sat, 13 Jul 2013 06:34:19 -0400 "Justin Piszcz" <jpiszcz@lucidpixels.com>
> > wrote:
> >
> >> Hello,
> >>
> >> Running 3.10 and I see the following for an md-raid1 of two SSDs:
> >>
> >> Checking /sys/block/md1/queue:
> >> add_random: 0
> >> discard_granularity: 512
> >> discard_max_bytes: 2147450880
> >> discard_zeroes_data: 0
> >> hw_sector_size: 512
> >> iostats: 0
> >> logical_block_size: 512
> >> max_hw_sectors_kb: 32767
> >> max_integrity_segments: 0
> >> max_sectors_kb: 512
> >> max_segment_size: 65536
> >> max_segments: 168
> >> minimum_io_size: 512
> >> nomerges: 0
> >> nr_requests: 128
> >> optimal_io_size: 0
> >> physical_block_size: 512
> >> read_ahead_kb: 8192
> >> rotational: 1
> >> rq_affinity: 0
> >> scheduler: none
> >> write_same_max_bytes: 0
> >>
> >> What should be seen:
> >> rotational: 0
> >
> > What has "rotational" got to do with "supports discard"?
> > There may be some correlation, but it isn't causal.
> >
> >> And possibly:
> >> discard_zeroes_data: 1
> >
> > This should be set as the 'or' of the same value from component devices.  And
> > does not enable or disable the use of discard.
> >
> > I don't think that "does this device support discard" appears in sysfs.
> >
> > I believe trim does work on md/raid1 if the underlying devices all support it.
> >
> > NeilBrown
> >
> >
> >
> >>
> >> Can anyone confirm if there is a workaround to allow TRIM when using
> >> md-raid1?
> >>
> >> Some related discussion here:
> >> http://us.generation-nt.com/answer/md-rotational-attribute-help-206571222.ht
> >> ml
> >> http://www.progtown.com/topic343938-ssd-strange-itself-conducts.html
> >>
> >>
> >> Justin.
> >>
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> 
> 
> 


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]