raw aio write guarantee

raw aio write guarantee

[iodophlymiaelo]

Hi all,

Just a quick question: How can a user-mode application ensure that an
AIO write on a raw block device (i.e. open()ed with O_DIRECT) has
really -really- been written to the disk and not residing in an on-disk
write cache where it could be lost in case of a power failure? Is
physical write guaranteed to have taken place before the AIO
completion? Or is there a system call that sends a cache flush
command to the drive (and waits for the drive to commit)?

Re: raw aio write guarantee

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

On Wed, 28 Sep 2005 00:12:33 PDT, iodophlymiaelo@gmail.com said:

> Just a quick question: How can a user-mode application ensure that an
> AIO write on a raw block device (i.e. open()ed with O_DIRECT) has
> really -really- been written to the disk and not residing in an on-disk
> write cache where it could be lost in case of a power failure?

Step 1: Make sure you buy disk drives that don't lie through their teeth
regarding details such as "is the write cache enabled?" or "did the flush
cache work?".  Such hardware can generate vast amounts of bad karma....

Step 2: Buy a UPS.  A good one.  You can't lose the cache during a power failure
that doesn't actually hit.

Step 3: http://catb.org/~esr/jargon/html/M/molly-guard.html - You want these.
Lots of them. 


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: raw aio write guarantee

[iodophlymiaelo]

On 9/28/05, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote:
> On Wed, 28 Sep 2005 00:12:33 PDT, iodophlymiaelo@gmail.com said:
>
> > Just a quick question: How can a user-mode application ensure that an
> > AIO write on a raw block device (i.e. open()ed with O_DIRECT) has
> > really -really- been written to the disk and not residing in an on-disk
> > write cache where it could be lost in case of a power failure?
>
> Step 1: Make sure you buy disk drives that don't lie through their teeth
> regarding details such as "is the write cache enabled?" or "did the flush
> cache work?".  Such hardware can generate vast amounts of bad karma....
>
> Step 2: Buy a UPS.  A good one.  You can't lose the cache during a power failure
> that doesn't actually hit.
>
> Step 3: http://catb.org/~esr/jargon/html/M/molly-guard.html - You want these.
> Lots of them.
>

I was asking what a user-application can do to prevent data loss, not
an application-user.

Re: raw aio write guarantee

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

On Wed, 28 Sep 2005 01:56:05 PDT, iodophlymiaelo@gmail.com said:

> I was asking what a user-application can do to prevent data loss, not
> an application-user.

Right.  However, if you actually care about the distinction between "made it
to the disk cache" and "made it to the platter", those are things you'll
want to address - in particular, if you have one of the evil disk drives
I mentioned, there's very little a user application can do to work around it.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: raw aio write guarantee

[iodophlymiaelo]

On 9/28/05, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote:
> On Wed, 28 Sep 2005 01:56:05 PDT, iodophlymiaelo@gmail.com said:
>
> > I was asking what a user-application can do to prevent data loss, not
> > an application-user.
>
> Right.  However, if you actually care about the distinction between "made it
> to the disk cache" and "made it to the platter", those are things you'll
> want to address - in particular, if you have one of the evil disk drives
> I mentioned, there's very little a user application can do to work around it.
>

Erm, is the hardware problem really as great as you're implying? Have you
personally encountered any bad drives made by reputable brands? Mostly I've
only heard only of people crying wolf and then realizing it was a problem with
their reasoning or with the assumption that fsync() actually works properly on
kernel X, where X doesn't even have to be that ancient a version of linux ;-)

Re: raw aio write guarantee

[Kyle Moffett]

On Sep 28, 2005, at 12:42:50, iodophlymiaelo@gmail.com wrote:
> Erm, is the hardware problem really as great as you're implying?  
> Have you personally encountered any bad drives made by reputable  
> brands? Mostly I've only heard only of people crying wolf and then  
> realizing it was a problem with their reasoning or with the  
> assumption that fsync() actually works properly on kernel X, where  
> X doesn't even have to be that ancient a version of linux ;-)

No, I've seen several sample cases on this list of drives where the  
IDE or SCSI cache flush commands did not trigger any disk activity  
and the only way to force it out of the cache was to write several  
meg of garbage to some file.  In some cases (Like RAID cards with  
good battery backup, for example), they may ignore the flush cache  
command as it isn't really useful (although on such good cards, they   
usually haw a way to turn it off, too).

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L++++(+ 
++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+++ t+(+ 
++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  !y?(-)
------END GEEK CODE BLOCK------