* Disabling barriers on NVC-backed HDD
@ 2017-11-15 17:01 Gionatan Danti
2017-11-15 17:31 ` Darrick J. Wong
0 siblings, 1 reply; 7+ messages in thread
From: Gionatan Danti @ 2017-11-15 17:01 UTC (permalink / raw)
To: linux-xfs; +Cc: g.danti
Hi list,
I'm not sure this is the correct list to post my question; if so, feel
free to ignore this message.
On XFS FAQs I (rightfully) read that barrier should be *always* enabled
(and they are, by default), unless a write-loss protected writeback
cache is used at the RAID card level. So far, so good.
Newer HDDs has themselves have a non-volatile cache (NVC) of some sort,
basically:
- DRAM + eMCL: in a powerloss event, the DRAM cache is immediately
dumped on the eMLC flash;
- DRAM + NOR flash: NOR mirrors a (small) portion of the DRAM cache,
used for write acceleration purpose;
- dedicated "dump" areas on the disks: they effectively mirror a portion
of the DRAM cache.
My question is: do you think it is safe to disable barriers, both for
XFS and in general terms, on these disks? Or they should be considered
as the same "dumb" unprotected DRAM caches found on classical HDD?
From a side, these *are* powerloss-protected caches. Problem is that all
these power-protection schemes are considered "secret sauce / trade
secret" by HDD vendors and, for this reason, there are very little (if
any) informations on their inner working.
Regards.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Disabling barriers on NVC-backed HDD
2017-11-15 17:01 Disabling barriers on NVC-backed HDD Gionatan Danti
@ 2017-11-15 17:31 ` Darrick J. Wong
2017-11-15 18:53 ` Gionatan Danti
2017-11-15 19:47 ` Christoph Hellwig
0 siblings, 2 replies; 7+ messages in thread
From: Darrick J. Wong @ 2017-11-15 17:31 UTC (permalink / raw)
To: Gionatan Danti; +Cc: linux-xfs
On Wed, Nov 15, 2017 at 06:01:17PM +0100, Gionatan Danti wrote:
> Hi list,
> I'm not sure this is the correct list to post my question; if so, feel free
> to ignore this message.
>
> On XFS FAQs I (rightfully) read that barrier should be *always* enabled (and
> they are, by default), unless a write-loss protected writeback cache is used
> at the RAID card level. So far, so good.
>
> Newer HDDs has themselves have a non-volatile cache (NVC) of some sort,
> basically:
> - DRAM + eMCL: in a powerloss event, the DRAM cache is immediately dumped on
> the eMLC flash;
> - DRAM + NOR flash: NOR mirrors a (small) portion of the DRAM cache, used
> for write acceleration purpose;
> - dedicated "dump" areas on the disks: they effectively mirror a portion of
> the DRAM cache.
>
> My question is: do you think it is safe to disable barriers, both for XFS
> and in general terms, on these disks? Or they should be considered as the
> same "dumb" unprotected DRAM caches found on classical HDD?
>
> From a side, these *are* powerloss-protected caches. Problem is that all
> these power-protection schemes are considered "secret sauce / trade secret"
> by HDD vendors and, for this reason, there are very little (if any)
> informations on their inner working.
I would evaluate these drives to find out if you really /can/ yank the
power without losing anything.
That said, if the manufacturers aren't willing to tell you how that
feature works, I'd just as soon pretend the feature didn't exist and
continue sending flushes to the drive.
FWIW if the drive really /does/ have a non-volatile WC then a flush
should have nearly zero overhead. (Or so you'd think...)
--D
> Regards.
>
> --
> Danti Gionatan
> Supporto Tecnico
> Assyoma S.r.l. - www.assyoma.it
> email: g.danti@assyoma.it - info@assyoma.it
> GPG public key ID: FF5F32A8
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Disabling barriers on NVC-backed HDD
2017-11-15 17:31 ` Darrick J. Wong
@ 2017-11-15 18:53 ` Gionatan Danti
2017-11-15 19:47 ` Christoph Hellwig
1 sibling, 0 replies; 7+ messages in thread
From: Gionatan Danti @ 2017-11-15 18:53 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs, g.danti
Il 15-11-2017 18:31 Darrick J. Wong ha scritto:
>
> I would evaluate these drives to find out if you really /can/ yank the
> power without losing anything.
If I can get some hard drives for testing, I'll surely do that. However,
any in-house testing inevitably has limited scope and it can miss some
important failure modes, so I am interested in other users' first-hand
experience.
> That said, if the manufacturers aren't willing to tell you how that
> feature works, I'd just as soon pretend the feature didn't exist and
> continue sending flushes to the drive.
From a point, I agree: I do not like black boxes. On the other hand,
hardware RAID controllers *are* black boxes, and we all use them...
> FWIW if the drive really /does/ have a non-volatile WC then a flush
> should have nearly zero overhead. (Or so you'd think...)
From my understanding (and I can be wrong), these drives honour cache
flushes, with the associated performance drop.
Ok, too much conjectures from my side, I think. I should *really* get my
hand on some of these drives...
Regards.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Disabling barriers on NVC-backed HDD
2017-11-15 17:31 ` Darrick J. Wong
2017-11-15 18:53 ` Gionatan Danti
@ 2017-11-15 19:47 ` Christoph Hellwig
2017-11-15 20:17 ` Gionatan Danti
1 sibling, 1 reply; 7+ messages in thread
From: Christoph Hellwig @ 2017-11-15 19:47 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Gionatan Danti, linux-xfs
On Wed, Nov 15, 2017 at 09:31:00AM -0800, Darrick J. Wong wrote:
> I would evaluate these drives to find out if you really /can/ yank the
> power without losing anything.
>
> That said, if the manufacturers aren't willing to tell you how that
> feature works, I'd just as soon pretend the feature didn't exist and
> continue sending flushes to the drive.
>
> FWIW if the drive really /does/ have a non-volatile WC then a flush
> should have nearly zero overhead. (Or so you'd think...)
And in that case it will report WCE=0 and Linux won't even flush.
As is the case for typical enterprise disks.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Disabling barriers on NVC-backed HDD
2017-11-15 19:47 ` Christoph Hellwig
@ 2017-11-15 20:17 ` Gionatan Danti
2017-11-15 20:18 ` Gionatan Danti
0 siblings, 1 reply; 7+ messages in thread
From: Gionatan Danti @ 2017-11-15 20:17 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Darrick J. Wong, linux-xfs, g.danti
Il 15-11-2017 20:47 Christoph Hellwig ha scritto:
> And in that case it will report WCE=0 and Linux won't even flush.
> As is the case for typical enterprise disks.
Good point. Based on what I read here[1], page n.4, it *seems* that
Seagate AWC (advanced write cache) enabled drives report WCD (write
cache disabled) to the host OS.
I think this is the key parameter to watch: if the HDD vendor is
confident enough to report WCE=0/WCD, than it should be safe running
without barriers (after all, in this case Linux *will* disable barrier
by default).
Thank you Christoph for pointing that.
Regards.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Disabling barriers on NVC-backed HDD
2017-11-15 20:17 ` Gionatan Danti
@ 2017-11-15 20:18 ` Gionatan Danti
2017-11-15 20:27 ` Gionatan Danti
0 siblings, 1 reply; 7+ messages in thread
From: Gionatan Danti @ 2017-11-15 20:18 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Darrick J. Wong, linux-xfs, g.danti
Il 15-11-2017 21:17 Gionatan Danti ha scritto:
> Il 15-11-2017 20:47 Christoph Hellwig ha scritto:
>> And in that case it will report WCE=0 and Linux won't even flush.
>> As is the case for typical enterprise disks.
>
> Good point. Based on what I read here[1], page n.4, it *seems* that
> Seagate AWC (advanced write cache) enabled drives report WCD (write
> cache disabled) to the host OS.
>
> I think this is the key parameter to watch: if the HDD vendor is
> confident enough to report WCE=0/WCD, than it should be safe running
> without barriers (after all, in this case Linux *will* disable barrier
> by default).
>
> Thank you Christoph for pointing that.
> Regards.
Sorry, I forgot the link:
https://www.seagate.com/files/www-content/product-content/enterprise-performance-savvio-fam/enterprise-performance-15k-hdd/_cross-product/_shared/doc/enchanced-cache-advantage-tp691.1-1610us.pdf
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Disabling barriers on NVC-backed HDD
2017-11-15 20:18 ` Gionatan Danti
@ 2017-11-15 20:27 ` Gionatan Danti
0 siblings, 0 replies; 7+ messages in thread
From: Gionatan Danti @ 2017-11-15 20:27 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Darrick J. Wong, linux-xfs, g.danti
Il 15-11-2017 21:18 Gionatan Danti ha scritto:
> Sorry, I forgot the link:
> https://www.seagate.com/files/www-content/product-content/enterprise-performance-savvio-fam/enterprise-performance-15k-hdd/_cross-product/_shared/doc/enchanced-cache-advantage-tp691.1-1610us.pdf
Ok, it seems that for Seagate drives the real document to read is that
one[1]. Quoting:
"When WCE=0, Advanced provides NVC-protected write caching over the
portion of the DRAM used to coalesce writes. Write data only goes into
NVC when there is an unexpected power loss to the drive. The NVC has
90-day data retention. When WCE=1, a Advanced Caching drive will operate
on writes like a standard drive–writes in cache are not protected by NVC
and may be lost with power loss."
In short:
- WCE=1 -> entire cache enabled -> powerloss = dataloss;
- WCE=0 -> small write cache segment enabled -> powerloss = nodataloss.
Unfortunately, I can not find anything on that regard about HGST's
MediaCache...
Regards.
[1]
https://www.seagate.com/www-content/product-content/enterprise-performance-savvio-fam/ent-perf-10k-hdd-v9-skybolt/en-us/docs/100818015c.pdf
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-11-15 20:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-15 17:01 Disabling barriers on NVC-backed HDD Gionatan Danti
2017-11-15 17:31 ` Darrick J. Wong
2017-11-15 18:53 ` Gionatan Danti
2017-11-15 19:47 ` Christoph Hellwig
2017-11-15 20:17 ` Gionatan Danti
2017-11-15 20:18 ` Gionatan Danti
2017-11-15 20:27 ` Gionatan Danti
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.