Re: libata in 2.4.24?

Re: libata in 2.4.24?

[Xose Vazquez Perez]

Marcelo Tosatti wrote:

> On Sat, 29 Nov 2003, Marcelo Tosatti wrote:
>>
>> I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
>> stable series. ;)

> I thought a bit more about this issue and I have a different opinion now.
>
> 2.6 is getting more and more stable and already includes libata --- users 
> who need it should use 2.6.

Does it mean that 2.4.x is going to freeze, and only critical and security
patches are going to be applied ?

Re: libata in 2.4.24?

[Marcelo Tosatti]

On Mon, 1 Dec 2003, Xose Vazquez Perez wrote:

> Marcelo Tosatti wrote:
> 
> > On Sat, 29 Nov 2003, Marcelo Tosatti wrote:
> >>
> >> I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
> >> stable series. ;)
> 
> > I thought a bit more about this issue and I have a different opinion now.
> >
> > 2.6 is getting more and more stable and already includes libata --- users 
> > who need it should use 2.6.
> 
> Does it mean that 2.4.x is going to freeze, and only critical and security
> patches are going to be applied ?

Yes this will happen in the near future.

I still might accept some "non critical" modifications (which is btw, not
an objective defition) to 2.4.24, but for 2.4.25 that will be the rule.

Re: libata in 2.4.24?

[Stephan von Krawczynski]

On Mon, 1 Dec 2003 12:11:00 -0200 (BRST)
Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:

> 
> 
> On Mon, 1 Dec 2003, Xose Vazquez Perez wrote:
> 
> > Marcelo Tosatti wrote:
> > 
> > > On Sat, 29 Nov 2003, Marcelo Tosatti wrote:
> > >>
> > >> I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
> > >> stable series. ;)
> > 
> > > I thought a bit more about this issue and I have a different opinion now.
> > >
> > > 2.6 is getting more and more stable and already includes libata --- users 
> > > who need it should use 2.6.
> > 
> > Does it mean that 2.4.x is going to freeze, and only critical and security
> > patches are going to be applied ?
> 
> Yes this will happen in the near future.

Please include Cyclades PC300 drivers before that ;-)

Regards,
Stephan

Re: libata in 2.4.24?

[bill davidsen]

In article <Pine.LNX.4.44.0312011209000.13692-100000@logos.cnet>,
Marcelo Tosatti  <marcelo.tosatti@cyclades.com> wrote:
| 
| 
| On Mon, 1 Dec 2003, Xose Vazquez Perez wrote:
| 
| > Marcelo Tosatti wrote:
| > 
| > > On Sat, 29 Nov 2003, Marcelo Tosatti wrote:
| > >>
| > >> I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
| > >> stable series. ;)
| > 
| > > I thought a bit more about this issue and I have a different opinion now.
| > >
| > > 2.6 is getting more and more stable and already includes libata --- users 
| > > who need it should use 2.6.
| > 
| > Does it mean that 2.4.x is going to freeze, and only critical and security
| > patches are going to be applied ?
| 
| Yes this will happen in the near future.
| 
| I still might accept some "non critical" modifications (which is btw, not
| an objective defition) to 2.4.24, but for 2.4.25 that will be the rule.

I hope you will continue to allow changes like new drivers and the
like, there are many things in 2.6 which are not only not working but
unlikely to ever be fixed because they are "old technology" and have
been replaced by more interesting and less functional alternatives, but
which are highly useful for people who have to use what is available in
terms of hardware.

That's "what the client/boss what's to provide" as well as "what the
individual can afford." It's not just hobby stuff or old PCs being used,
there are people selling laptops with ACPI more disfunctional than the
Simpsons. Suspend via APM works fine, though.

Unless a miracle occurs it will take as long for 2.6 to be really stable
and fully functional as it did for 2.[024]. When it goes out in distros
and gets abused by users for a while the sharp corners will be broken off.
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: libata in 2.4.24?

[Jeff Garzik]

bill davidsen wrote:
> Unless a miracle occurs it will take as long for 2.6 to be really stable
> and fully functional as it did for 2.[024]. When it goes out in distros
> and gets abused by users for a while the sharp corners will be broken off.


I think that's a bit of an extremist view.

The 2.6 rpms Arjan has posted have gotten good use, and most people seem 
to think 2.6.0-test is a _lot_ more stable than 2.4.0 was at release time.

It's certainly true that a raft of currently-unknown bugs will show up 
when distros start shipping 2.6-based distros (I think the first was 
Mandrake, as an experimental extra, soon be followed by Fedora?).  Such 
bugs always appear.

I venture to say, for the vast majority of hardware, including my 28MB 
K6-2 laptop and my P133 firewall, 2.6 works just as well as 2.4 does 
currently.

	Jeff

Re: libata in 2.4.24?

[Bill Davidsen]

On Wed, 3 Dec 2003, Jamie Lokier wrote:

> bill davidsen wrote:
> > With O_SYNC files there is the possibility of having a don't cache bit
> > in the packet to the drive, even with write caching. With fsync I don't
> > see any way to do it after the fact for only some of the data in the
> > drive cache. That's just an observation.
> 
> With fsync, can't you write all the dirty pages with that bit set,
> write _again_ all the pages in RAM which are clean but which have
> never been written with the don't-cache bit, and read-then-write with
> the bit set all the pages which are not in RAM but which were dirtied
> and written without the don't cache bit set?

Actually, what I meant was to pass the bit to the drive, so that it would
not return completion status until physical completion. That can be done
for O_SYNC. But fsync() is after the fact, the o/s has no way of knowing
that the pages already sent to the drive have been written to media except
to do a cache flush and flush everything.

I don't think there's anything else you can do with fsync() with or
without the bit, since you may no longer have the buffers to send with the
bit set. Moreover, some drives, reportedly IBM, tend to botch a sevtor
being written during power fail, if I understand your proposal it *cold*
result in a buffer being sent to the drive twice, and a power fail during
the 2nd write could clobber the good data you wrote.

That's assuming I understand what you propose.

> 
> I know, it sounds a bit complicated :)
> 
> But would it work?

Maybe a guru will disagree, but I would say that just switching to what
SCSI does and caching the write on the drive and not returning done status
until it completes is the right solution. Maybe a drive vendor will do
that, maybe it's in SATA-2 spec, I was just making up the bit which
indicated a realtime write buffer, as a way O_SYNC could work without
killing performance.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: libata in 2.4.24?

[Jens Axboe]

On Tue, Dec 02 2003, Mike Fedyk wrote:
> On Tue, Dec 02, 2003 at 11:31:45AM -0500, Greg Stark wrote:
> > 
> > Mike Fedyk <mfedyk@matchmail.com> writes:
> > 
> > > > Libata, uses the scsi system instead of the existing ide layer because many
> > > > new sata controllers are using an interface that is very similair to scsi
> > > > (much like atapi).
> > 
> > Now I have a different question. Does the scsi-like SATA interface include tcq?
> > 
> > Because one of the long-standing issues with IDE drives and Postgres is the
> > fact that even after issuing an fsync the data may be sitting in the drive's
> > buffer. This doesn't happen with SCSI because the drives aren't forced to lie
> > about the data being on disk in order to handle subsequent requests. Turning
> > off write-caching on IDE drives absolutely destroys performance.
> 
> There are PATA drives that do TCQ too, but you have to look for that feature
> specifically.  IDE TCQ is in 2.6, but is still experemental.  I think Jens
> Axboe was the one working on it IIRC.  He would have more details.

Yes that was me. PATA TCQ is currently disabled in the 2.6 kernel
configs, and it will most likely just go away sometime soonish. It's not
a very useful feature, PATA TCQ is pretty broken. I knew this from the
beginning, but thought that it would be fun to try it in real life. It's
just not worth it, though.

SATA host queueing support will be useful.

-- 
Jens Axboe

Re: libata in 2.4.24?

[Jamie Lokier]

bill davidsen wrote:
> With O_SYNC files there is the possibility of having a don't cache bit
> in the packet to the drive, even with write caching. With fsync I don't
> see any way to do it after the fact for only some of the data in the
> drive cache. That's just an observation.

With fsync, can't you write all the dirty pages with that bit set,
write _again_ all the pages in RAM which are clean but which have
never been written with the don't-cache bit, and read-then-write with
the bit set all the pages which are not in RAM but which were dirtied
and written without the don't cache bit set?

I know, it sounds a bit complicated :)

But would it work?

-- Jamie

Re: libata in 2.4.24?

[Xose Vazquez Perez]

Jeff Garzik wrote:

> Sorry, SCSI is moving to point-to-point serial attachment (SAS), too :) 
>     While not quite ready to call Parallel SCSI dead, it's definitely 
> dying...

Intel has a nice paper [1] about disk interface technologies.
And Serial ATA and SCSI are the future.

[1] http://www.intel.com/technology/serialATA/pdf/NP2108.pdf - 68k

Re: libata in 2.4.24?

[Jeff Garzik]

bill davidsen wrote:
> Until multiple devices be string are available SATA will have logistical
> problems scaling. The small cable is an advantage running a few drives
> in a box, but a server with 40 drives or so would go from a cable bundle
> out the back, about 5cm by 1 cm, to a real bunch of those little round
> cables running everywhere. Certainly doable, but I think I'd name the
> server "Medusa" if I built it.


Sorry, SCSI is moving to point-to-point serial attachment (SAS), too :) 
    While not quite ready to call Parallel SCSI dead, it's definitely 
dying...

	Jeff

Re: libata in 2.4.24?

[Mike Fedyk]

On Tue, Dec 02, 2003 at 11:18:24PM +0000, Bill Davidsen wrote:
> In article <20031202230216.GB4154@mis-mike-wstn.matchmail.com>,
> Mike Fedyk  <mfedyk@matchmail.com> wrote:
> | On Tue, Dec 02, 2003 at 10:34:20PM +0000, Bill Davidsen wrote:
> | > after each O_SYNC write, so that's probably not practical. Clearly the
> | > best solution is a full SCSI implementation over PATA/SATA, but that
> | > would eliminate some of the justification for SCSI devices at premium
> | > prices.
> | 
> | In many ways, that is exactly what SATA is. :)
> 
> Until multiple devices be string are available SATA will have logistical
> problems scaling. The small cable is an advantage running a few drives
> in a box, but a server with 40 drives or so would go from a cable bundle
> out the back, about 5cm by 1 cm, to a real bunch of those little round
> cables running everywhere. Certainly doable, but I think I'd name the
> server "Medusa" if I built it.
> 

Isn't this what SSCSI (Serial SCSI) will be doing also -- one drive one cable?

I'd imagine that you'd have one connection to the enclosure, and the entire
enclosure will look like a single drive.  I've heard of SCSI enclosures like
this (even ones that hold ide drives, but talk scsi to the host).

> I believe SATA-2 will address this, if I may believe what's projected
> for an unwritten standard.

There will probably be many ways to get around any issues that may come up
with SATA until SATA-2 comes out.

Re: libata in 2.4.24?

[bill davidsen]

In article <20031202230216.GB4154@mis-mike-wstn.matchmail.com>,
Mike Fedyk  <mfedyk@matchmail.com> wrote:
| On Tue, Dec 02, 2003 at 10:34:20PM +0000, Bill Davidsen wrote:
| > after each O_SYNC write, so that's probably not practical. Clearly the
| > best solution is a full SCSI implementation over PATA/SATA, but that
| > would eliminate some of the justification for SCSI devices at premium
| > prices.
| 
| In many ways, that is exactly what SATA is. :)

Until multiple devices be string are available SATA will have logistical
problems scaling. The small cable is an advantage running a few drives
in a box, but a server with 40 drives or so would go from a cable bundle
out the back, about 5cm by 1 cm, to a real bunch of those little round
cables running everywhere. Certainly doable, but I think I'd name the
server "Medusa" if I built it.

I believe SATA-2 will address this, if I may believe what's projected
for an unwritten standard.
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: libata in 2.4.24?

[Mike Fedyk]

On Tue, Dec 02, 2003 at 10:34:20PM +0000, Bill Davidsen wrote:
> after each O_SYNC write, so that's probably not practical. Clearly the
> best solution is a full SCSI implementation over PATA/SATA, but that
> would eliminate some of the justification for SCSI devices at premium
> prices.

In many ways, that is exactly what SATA is. :)

Re: libata in 2.4.24?

[bill davidsen]

In article <877k1f9e1g.fsf@stark.dyndns.tv>,
Greg Stark  <gsstark@mit.edu> wrote:
| Jeff Garzik <jgarzik@pobox.com> writes:

| > > This doesn't happen with SCSI disks where multiple requests can be pending so
| > > there's no urgency to reporting a false success. The request doesn't complete
| > > until the write hits disk. As a result SCSI disks are reliable for database
| > > operation and IDE disks aren't unless write caching is disabled.
| > 
| > This is not really true.
| > 
| > Regardless of TCQ, if the OS driver has not issued a FLUSH CACHE (IDE)
| > or SYNCHRONIZE CACHE (SCSI), then the data is not guaranteed to be on
| > the disk media.  Plain and simple.
| 
| That doesn't agree with people's experience. People seem to find that SCSI
| drives never cache writes. This sort of makes sense since there's just not
| much reason to report a write success before the write can be performed.
| There's no performance advantage as long as more requests can be queued up.

I hope you mean the drives don't report completion until the data is on
the platter, clearly the data is cached in the drive until it can be
written.
| 
| 
| > If fsync(2) returns without a flush-cache, then your data is not
| > guaranteed to be on the disk.  And as you noted, flush-cache destroys
| > performance.
| 
| It's my understanding that it doesn't. There was some discussion in the past
| month about making the drivers issue syncs for journalled filesystems, but
| even then the idea of adding it to fsync or O_SYNC files wasn't the
| motivation.

With O_SYNC files there is the possibility of having a don't cache bit
in the packet to the drive, even with write caching. With fsync I don't
see any way to do it after the fact for only some of the data in the
drive cache. That's just an observation.

Clearly with a completion status coming back after actual completion
O_SYNC or fsync reduce to "wait for the ack from the drive."
| 
| 
| > There are three levels:
| > 
| > a) Data is successfully transferred to the controller/drive queue (TCQ).
| > b) Data is successfully transferred to the drive's internal buffers.
| > c) The drive successfully transfers data to the media.
| 
| Only the third is of interest to Postgres or other databases. In fact, I
| suspect only the third is of interest to other systems that are supposed to be
| reliable like MTAs etc. I think Wietse and others would be shocked if they
| were told fsync wasn't guaranteed to have waited until the writes had actually
| hit the media.

I think for reliability fsync has to flush cache, regardless of the
performance hit. I think a drive would be unusably slow if you did it
after each O_SYNC write, so that's probably not practical. Clearly the
best solution is a full SCSI implementation over PATA/SATA, but that
would eliminate some of the justification for SCSI devices at premium
prices.
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: libata in 2.4.24?

[Greg Stark]

Jeff Garzik <jgarzik@pobox.com> writes:

> Some IDE _and/or_ SCSI drives do not cache writes.  For these drives,
> the _absence_ of an OS flush-cache command still means your data gets
> to the platter.

In theory you could have an IDE drive that didn't cache writes, or a SCSI
drive that did. Except in practice it seems all IDE drives cache writes by
default and perform like dogs if they don't. And in practice all SCSI drives
appear to not cache writes and perform fine.

I guess my question is whether a new round of ATA drives will be coming out
where you can turn off write caching and still get decent performance because
the interface is more SCSI-like with deep enough queues. If so they'll
probably disable write caching altogether, but if they don't the user could
always do it.

And if such a new round of ATA drives will be coming out, exactly what should
I be watching for. SATA alone isn't enough, what featureset will this feature
come along with? 

> The core problem is not issuing a flush-cache command, it sounds like.
> The drive technology (wcache, or no) is largely irrelevant.

Well issuing a flush-cache is a much bigger performance hit than merely not
caching the writes in the first place. There could be lots of other writes to
other files in the system. In fact it's likely there are lots of other writes
to other files in postgres itself, most of the time it's only fsyncing the
transaction log.

-- 
greg

Re: libata in 2.4.24?

[Jeff Garzik]

On Tue, Dec 02, 2003 at 03:10:19PM -0500, Greg Stark wrote:
> Jeff Garzik <jgarzik@pobox.com> writes:
> 
> > So, today, no acknowledgement occurs until the data _really_ is in the
> > drive's buffers.
> 
> The drive's buffers isn't good enough. If power is lost the write will be lost
> and the database corrupt. It needs to be on the platters.

Certainly agreed.


> > > This doesn't happen with SCSI disks where multiple requests can be pending so
> > > there's no urgency to reporting a false success. The request doesn't complete
> > > until the write hits disk. As a result SCSI disks are reliable for database
> > > operation and IDE disks aren't unless write caching is disabled.
> > 
> > This is not really true.
> > 
> > Regardless of TCQ, if the OS driver has not issued a FLUSH CACHE (IDE)
> > or SYNCHRONIZE CACHE (SCSI), then the data is not guaranteed to be on
> > the disk media.  Plain and simple.
> 
> That doesn't agree with people's experience. People seem to find that SCSI
> drives never cache writes. This sort of makes sense since there's just not
> much reason to report a write success before the write can be performed.
> There's no performance advantage as long as more requests can be queued up.

Some IDE _and/or_ SCSI drives do not cache writes.  For these drives,
the _absence_ of an OS flush-cache command still means your data gets
to the platter.

The core problem is not issuing a flush-cache command, it sounds like.
The drive technology (wcache, or no) is largely irrelevant.


> > If fsync(2) returns without a flush-cache, then your data is not
> > guaranteed to be on the disk.  And as you noted, flush-cache destroys
> > performance.
> 
> It's my understanding that it doesn't. There was some discussion in the past

eh?  flush-cache very definitely hurts performance, on both IDE and
SCSI, for drives that support write caching.


> > There are three levels:
> > 
> > a) Data is successfully transferred to the controller/drive queue (TCQ).
> > b) Data is successfully transferred to the drive's internal buffers.
> > c) The drive successfully transfers data to the media.
> 
> Only the third is of interest to Postgres or other databases. In fact, I

Certainly.


> suspect only the third is of interest to other systems that are supposed to be
> reliable like MTAs etc. I think Wietse and others would be shocked if they
> were told fsync wasn't guaranteed to have waited until the writes had actually
> hit the media.

As well he should be :)

	Jeff

Re: libata in 2.4.24?

[Greg Stark]

Jeff Garzik <jgarzik@pobox.com> writes:

> So, today, no acknowledgement occurs until the data _really_ is in the
> drive's buffers.

The drive's buffers isn't good enough. If power is lost the write will be lost
and the database corrupt. It needs to be on the platters.


> > This doesn't happen with SCSI disks where multiple requests can be pending so
> > there's no urgency to reporting a false success. The request doesn't complete
> > until the write hits disk. As a result SCSI disks are reliable for database
> > operation and IDE disks aren't unless write caching is disabled.
> 
> This is not really true.
> 
> Regardless of TCQ, if the OS driver has not issued a FLUSH CACHE (IDE)
> or SYNCHRONIZE CACHE (SCSI), then the data is not guaranteed to be on
> the disk media.  Plain and simple.

That doesn't agree with people's experience. People seem to find that SCSI
drives never cache writes. This sort of makes sense since there's just not
much reason to report a write success before the write can be performed.
There's no performance advantage as long as more requests can be queued up.


> If fsync(2) returns without a flush-cache, then your data is not
> guaranteed to be on the disk.  And as you noted, flush-cache destroys
> performance.

It's my understanding that it doesn't. There was some discussion in the past
month about making the drivers issue syncs for journalled filesystems, but
even then the idea of adding it to fsync or O_SYNC files wasn't the
motivation.


> There are three levels:
> 
> a) Data is successfully transferred to the controller/drive queue (TCQ).
> b) Data is successfully transferred to the drive's internal buffers.
> c) The drive successfully transfers data to the media.

Only the third is of interest to Postgres or other databases. In fact, I
suspect only the third is of interest to other systems that are supposed to be
reliable like MTAs etc. I think Wietse and others would be shocked if they
were told fsync wasn't guaranteed to have waited until the writes had actually
hit the media.

-- 
greg

Re: libata in 2.4.24?

[Jeff Garzik]

On Tue, Dec 02, 2003 at 01:51:17PM -0500, Greg Stark wrote:
> 
> Jeff Garzik <jgarzik@pobox.com> writes:
> 
> > If true, this is an IDE driver bug...  assuming the drive itself
> > doesn't lie about FLUSH CACHE results (a few do).
> 
> I don't think the IDE drivers issue FLUSH CACHE after every write on O_SYNC,
> or after fsync calls. The "lying" discussed on the database lists is when a
> normal write is issued, IDE disks report immediate success even before the
> write hits disk. As far as I know from the lists it seems *all* IDE disks
> behave this way unless write caching is disabled.

The way CONFIG_IDE (the traditional IDE driver) and libata work right
now, when the drive indicates that the read/write is complete, the OS
driver indicates to the filesystem that the data transaction is
complete.

So, today, no acknowledgement occurs until the data _really_ is in the
drive's buffers.

That said, "the database lists" may be seeing page cache effects.
write(2) will certainly report success long before the data transaction
is even sent to the driver!  You must fsync(2) to flush data from the
page cache to the IDE driver.


> This doesn't happen with SCSI disks where multiple requests can be pending so
> there's no urgency to reporting a false success. The request doesn't complete
> until the write hits disk. As a result SCSI disks are reliable for database
> operation and IDE disks aren't unless write caching is disabled.

This is not really true.

Regardless of TCQ, if the OS driver has not issued a FLUSH CACHE (IDE)
or SYNCHRONIZE CACHE (SCSI), then the data is not guaranteed to be on
the disk media.  Plain and simple.

If fsync(2) returns without a flush-cache, then your data is not
guaranteed to be on the disk.  And as you noted, flush-cache destroys
performance.


> I'm unclear on which of your #2 or #3 will be the solution though. Do either
> or both of them require that writes actually hit disk before the drive reports
> success? Do either of them allow that semantic without destroying concurrent
> performance?

There are three levels:

a) Data is successfully transferred to the controller/drive queue (TCQ).
b) Data is successfully transferred to the drive's internal buffers.
c) The drive successfully transfers data to the media.

Acknowledgement of (a) is basically instantaneous.  The OS driver simply
adds a drive read/write command to a list that the host controller can
see.

Acknowledgement of (b) happens fairly rapidly, limited by the device's
throughput and seek times, internal buffer load (amount of work todo),
and internal algorithms.

Acknowledgement of (c) _never_ occurs.  One must issue the flush-cache
drive command to be certain that the drive has flushed its write
buffers.

	Jeff

Re: libata in 2.4.24?

[Greg Stark]

Jeff Garzik <jgarzik@pobox.com> writes:

> If true, this is an IDE driver bug...  assuming the drive itself
> doesn't lie about FLUSH CACHE results (a few do).

I don't think the IDE drivers issue FLUSH CACHE after every write on O_SYNC,
or after fsync calls. The "lying" discussed on the database lists is when a
normal write is issued, IDE disks report immediate success even before the
write hits disk. As far as I know from the lists it seems *all* IDE disks
behave this way unless write caching is disabled.

This doesn't happen with SCSI disks where multiple requests can be pending so
there's no urgency to reporting a false success. The request doesn't complete
until the write hits disk. As a result SCSI disks are reliable for database
operation and IDE disks aren't unless write caching is disabled.

> If the drive lies, there isn't a darned thing we can do about it...
>
> If the drive lies, there isn't a darned thing the controller can do
> about it, either ;-)

Of course not, but the assumption is that they're only lying because of the
lack of a TCQ interface. If they could have multiple requests pending there
would be no need to lie any more at all.

I'm unclear on which of your #2 or #3 will be the solution though. Do either
or both of them require that writes actually hit disk before the drive reports
success? Do either of them allow that semantic without destroying concurrent
performance?

-- 
greg

Re: libata in 2.4.24?

[Jeff Garzik]

On Tue, Dec 02, 2003 at 10:46:48AM -0800, Mike Fedyk wrote:
> On Tue, Dec 02, 2003 at 01:04:58PM -0500, Jeff Garzik wrote:
> > On Tue, Dec 02, 2003 at 09:40:48AM -0800, Mike Fedyk wrote:
> > > There are PATA drives that do TCQ too, but you have to look for that feature
> > > specifically.  IDE TCQ is in 2.6, but is still experemental.  I think Jens
> > > Axboe was the one working on it IIRC.  He would have more details.
> > 
> > Let us distinguish three types of TCQ:
> > 1) PATA drive-side TCQ (now called "legacy TCQ")
> > 2) Controller-side TCQ
> > 3) SATA drive/controller-side TCQ ("first party DMA")
> > 
> > libata will never support #1, which is what 2.6 supports in experimental
> > option.
> 
> An experemental option with the ide layer, not libata, right?

Correct.


> > libata will support #2 very soon, and will support #3 when hardware is
> > available.
> > 
> 
> If you have Controller-side TCQ, then will it work with any IDE PATA/SATA
> drive?

Yes.

	Jeff

Re: libata in 2.4.24?

[Mike Fedyk]

On Tue, Dec 02, 2003 at 01:04:58PM -0500, Jeff Garzik wrote:
> On Tue, Dec 02, 2003 at 09:40:48AM -0800, Mike Fedyk wrote:
> > There are PATA drives that do TCQ too, but you have to look for that feature
> > specifically.  IDE TCQ is in 2.6, but is still experemental.  I think Jens
> > Axboe was the one working on it IIRC.  He would have more details.
> 
> Let us distinguish three types of TCQ:
> 1) PATA drive-side TCQ (now called "legacy TCQ")
> 2) Controller-side TCQ
> 3) SATA drive/controller-side TCQ ("first party DMA")
> 
> libata will never support #1, which is what 2.6 supports in experimental
> option.

An experemental option with the ide layer, not libata, right?

> 
> libata will support #2 very soon, and will support #3 when hardware is
> available.
> 

If you have Controller-side TCQ, then will it work with any IDE PATA/SATA
drive?

> > > Do the new SATA drives and controllers provide a solution to this?
> > 
> > It's not SATA specific, and I'm not sure if any ide controller can support
> > TCQ or if only a specific list are compatible.
> 
> The TCQ you are thinking of has been deprecated by the people who make
> IDE drives ;-)

Ahh, it's good to know that our TCQ support isn't lagging.  We're even
supporting TCQ standards that were only in place for 1 or 2 years. :)

Re: libata in 2.4.24?

[Jeff Garzik]

On Tue, Dec 02, 2003 at 09:40:48AM -0800, Mike Fedyk wrote:
> There are PATA drives that do TCQ too, but you have to look for that feature
> specifically.  IDE TCQ is in 2.6, but is still experemental.  I think Jens
> Axboe was the one working on it IIRC.  He would have more details.

Let us distinguish three types of TCQ:
1) PATA drive-side TCQ (now called "legacy TCQ")
2) Controller-side TCQ
3) SATA drive/controller-side TCQ ("first party DMA")

libata will never support #1, which is what 2.6 supports in experimental
option.

libata will support #2 very soon, and will support #3 when hardware is
available.


> > Do the new SATA drives and controllers provide a solution to this?
> 
> It's not SATA specific, and I'm not sure if any ide controller can support
> TCQ or if only a specific list are compatible.

The TCQ you are thinking of has been deprecated by the people who make
IDE drives ;-)

	Jeff

Re: libata in 2.4.24?

[Jeff Garzik]

On Tue, Dec 02, 2003 at 11:31:45AM -0500, Greg Stark wrote:
> 
> Mike Fedyk <mfedyk@matchmail.com> writes:
> 
> > > Libata, uses the scsi system instead of the existing ide layer because many
> > > new sata controllers are using an interface that is very similair to scsi
> > > (much like atapi).
> 
> Now I have a different question. Does the scsi-like SATA interface include tcq?

Yes, it does.  But it depends on whether or not the host controller
supports TCQ.


> Because one of the long-standing issues with IDE drives and Postgres is the
> fact that even after issuing an fsync the data may be sitting in the drive's
> buffer.

If true, this is an IDE driver bug...  assuming the drive itself
doesn't lie about FLUSH CACHE results (a few do).


> This doesn't happen with SCSI because the drives aren't forced to lie
> about the data being on disk in order to handle subsequent requests. Turning
> off write-caching on IDE drives absolutely destroys performance.

If the drive lies, there isn't a darned thing we can do about it...


> Do the new SATA drives and controllers provide a solution to this?

If the drive lies, there isn't a darned thing the controller can do
about it, either ;-)

	Jeff

Re: libata in 2.4.24?

[Mike Fedyk]

On Tue, Dec 02, 2003 at 11:31:45AM -0500, Greg Stark wrote:
> 
> Mike Fedyk <mfedyk@matchmail.com> writes:
> 
> > > Libata, uses the scsi system instead of the existing ide layer because many
> > > new sata controllers are using an interface that is very similair to scsi
> > > (much like atapi).
> 
> Now I have a different question. Does the scsi-like SATA interface include tcq?
> 
> Because one of the long-standing issues with IDE drives and Postgres is the
> fact that even after issuing an fsync the data may be sitting in the drive's
> buffer. This doesn't happen with SCSI because the drives aren't forced to lie
> about the data being on disk in order to handle subsequent requests. Turning
> off write-caching on IDE drives absolutely destroys performance.

There are PATA drives that do TCQ too, but you have to look for that feature
specifically.  IDE TCQ is in 2.6, but is still experemental.  I think Jens
Axboe was the one working on it IIRC.  He would have more details.

> 
> Do the new SATA drives and controllers provide a solution to this?

It's not SATA specific, and I'm not sure if any ide controller can support
TCQ or if only a specific list are compatible.

Mike

Re: libata in 2.4.24?

[Greg Stark]

Mike Fedyk <mfedyk@matchmail.com> writes:

> > Libata, uses the scsi system instead of the existing ide layer because many
> > new sata controllers are using an interface that is very similair to scsi
> > (much like atapi).

Now I have a different question. Does the scsi-like SATA interface include tcq?

Because one of the long-standing issues with IDE drives and Postgres is the
fact that even after issuing an fsync the data may be sitting in the drive's
buffer. This doesn't happen with SCSI because the drives aren't forced to lie
about the data being on disk in order to handle subsequent requests. Turning
off write-caching on IDE drives absolutely destroys performance.

Do the new SATA drives and controllers provide a solution to this?

-- 
greg

Re: libata in 2.4.24?

[Mike Fedyk]

On Mon, Dec 01, 2003 at 09:53:36PM -0800, Mike Fedyk wrote:
> On Tue, Dec 02, 2003 at 12:36:19AM -0500, Greg Stark wrote:
> > 
> > Erik Steffl <steffl@bigfoot.com> writes:
> > 
> > >    not for drives >133GB (I have intel D865PERL mb and 250GB matrox, it doesn't
> > > work without SCSI_ATA (at all), it cannot read/write above 133GB without libata
> > > patches)
> > 
> > My ICH5 was happily using my brand new 200GB SATA maxtor drive, at least up
> > until last Friday when it crashed. Grumble, a brand new drive.
> > 
> > I guess what I'm looking for is the "FAQ" or "README" file that most projects
> > have. What are the advantages of using the libata patch vs the stock drivers
> > and vice versa? What are the differences between the two?
> 
> Libata, uses the scsi system instead of the existing ide layer because many
> new sata controllers are using an interface that is very similair to scsi
> (much like atapi).
> 
> The existing ide layer is geared tward pata (old style with 40 or 80 pins),
> and any sata controller that behaves like a pata coltroller.  That's why
> ICH5 and VIA's controllers are supported first (and not with all of their
> features like hotplug and some others).
> 
> I'd say use the stock ide driver unless libata gives you features you need,
> or a >30% speed boost.
> 

Re: libata in 2.4.24?

[Greg Stark]

Erik Steffl <steffl@bigfoot.com> writes:

>    not for drives >133GB (I have intel D865PERL mb and 250GB matrox, it doesn't
> work without SCSI_ATA (at all), it cannot read/write above 133GB without libata
> patches)

My ICH5 was happily using my brand new 200GB SATA maxtor drive, at least up
until last Friday when it crashed. Grumble, a brand new drive.

I guess what I'm looking for is the "FAQ" or "README" file that most projects
have. What are the advantages of using the libata patch vs the stock drivers
and vice versa? What are the differences between the two?

-- 
greg

Re: libata in 2.4.24?

[Samuel Flory]

Greg Stark wrote:
> Samuel Flory <sflory@rackable.com> writes:
> 
> 
>>   What chipset are you using?  Assumming that hda is your sata drive. What are
>>the results of the following "hdarm -t /dev/hda" "hdparm -dvi /dev/hda"   The
>>ICH5 chipset is the only chipset I've found that works well without libata.
> 
> 
> Ah, my motherboard is in fact an ICH5 I believe.
> Incidentally my kernel is actually 2.4.23-pre4.
> 

   Generally the ICH5 support just works, but it may not be doing dma. 
I've have some stability issues with some configs before I started 
applying libata patches to my kernel.  This was a long time ago.  Also 
ctcs pounds on things pretty hard.

http://marc.theaimsgroup.com/?l=linux-kernel&m=105666857613064&w=2

> Is there any documentation about what libata is and what it does differently
> from the stock kernel? 

   Libata is just another driver patch for the linux kernel.  It uses 
the scsi subsystem instead of the ide subsystem.  Jeff has pdf some 
where, but as I remember it was geared for developers.

> Why is it being developed separately instead of as a
> set of new drivers in the kernel like normal? 
> 

   What? Most drivers are developed outside the main kernel tree. 
Marcelo's job is to stop people from doing that kind of thing in the 
stable tree;-)

-- 
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@rackable.com>

Re: libata in 2.4.24?

[Jeff Garzik]

Greg Stark wrote:
> Is there any documentation about what libata is and what it does differently
> from the stock kernel? Why is it being developed separately instead of as a

Nothing "different" from the stock kernel; this is how all drivers are 
developed.  libata is the Serial ATA driver for Linux.  Some chipsets -- 
ICH5 and VIA SATA notably -- look so much like PATA that it's easy to 
let the existing drivers/ide driver use them.  You won't get SATA 
hotplug or similar SATA-only features, but as long as you can access 
your SATA hard drive, who cares?  :)

libata is in the "stock" 2.6 kernel, FWIW, too.

	Jeff

Re: libata in 2.4.24?

[Erik Steffl]

Samuel Flory wrote:
> Greg Stark wrote:
> 
>> Samuel Flory <sflory@rackable.com> writes:
>>
>>
>>>   It's getting harder to find a newly released motherboard without 
>>> onboard
>>> sata.  Not having  libata support means that anyone running 2.4 
>>> unpatched will
>>> be unable to use such systems.
>>
>>
>>
>> My motherboard has a SATA controller and I'm using it in non-legacy 
>> mode with
>> 2.4.23. What is this libata thing I'm supposed to be needing?
>>
> 
>   What chipset are you using?  Assumming that hda is your sata drive. 
> What are the results of the following "hdarm -t /dev/hda" "hdparm -dvi 
> /dev/hda"   The ICH5 chipset is the only chipset I've found that works 
> well without libata.

   not for drives >133GB (I have intel D865PERL mb and 250GB matrox, it 
doesn't work without SCSI_ATA (at all), it cannot read/write above 133GB 
without libata patches)

	erik

Re: libata in 2.4.24?

[Greg Stark]

Samuel Flory <sflory@rackable.com> writes:

>    What chipset are you using?  Assumming that hda is your sata drive. What are
> the results of the following "hdarm -t /dev/hda" "hdparm -dvi /dev/hda"   The
> ICH5 chipset is the only chipset I've found that works well without libata.

Ah, my motherboard is in fact an ICH5 I believe.
Incidentally my kernel is actually 2.4.23-pre4.

Is there any documentation about what libata is and what it does differently
from the stock kernel? Why is it being developed separately instead of as a
set of new drivers in the kernel like normal? 

-- 
greg

Re: libata in 2.4.24?

[Justin Cormack]

On Mon, 2003-12-01 at 21:12, Greg Stark wrote:
> 
> Samuel Flory <sflory@rackable.com> writes:
> 
> >    It's getting harder to find a newly released motherboard without onboard
> > sata.  Not having  libata support means that anyone running 2.4 unpatched will
> > be unable to use such systems.
> 
> My motherboard has a SATA controller and I'm using it in non-legacy mode with
> 2.4.23. What is this libata thing I'm supposed to be needing?

It depends which sata controller it is whether it works. SOme do some
dont...

Justin

Re: libata in 2.4.24?

[Samuel Flory]

Greg Stark wrote:
> Samuel Flory <sflory@rackable.com> writes:
> 
> 
>>   It's getting harder to find a newly released motherboard without onboard
>>sata.  Not having  libata support means that anyone running 2.4 unpatched will
>>be unable to use such systems.
> 
> 
> My motherboard has a SATA controller and I'm using it in non-legacy mode with
> 2.4.23. What is this libata thing I'm supposed to be needing?
> 

   What chipset are you using?  Assumming that hda is your sata drive. 
What are the results of the following "hdarm -t /dev/hda" "hdparm -dvi 
/dev/hda"   The ICH5 chipset is the only chipset I've found that works 
well without libata.


-- 
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@rackable.com>

Re: libata in 2.4.24?

[Greg Stark]

Samuel Flory <sflory@rackable.com> writes:

>    It's getting harder to find a newly released motherboard without onboard
> sata.  Not having  libata support means that anyone running 2.4 unpatched will
> be unable to use such systems.

My motherboard has a SATA controller and I'm using it in non-legacy mode with
2.4.23. What is this libata thing I'm supposed to be needing?

-- 
greg

Re: libata in 2.4.24?

[Samuel Flory]

Marcelo Tosatti wrote:
> 
> On Sat, 29 Nov 2003, Marcelo Tosatti wrote:
> 
> 
>>
>>On Sat, 29 Nov 2003, Samuel Flory wrote:
>>
>>
>>>   Are you considering including libata support for 2.4.24?  From my 
>>>testing with a number of different embedded sata chipsets (mostly ICH, 
>>>SI, and Promise) it appears very stable.  I'm not seeing any data 
>>>corruption or lockups when running Cerberus with 2.4.23-rc5 + libata 
>>>patch.  The only troubles I've had were with initialization of embedded 
>>>promise sata controllers. (I still need to test with Jeff's latest fixes 
>>>for this.)
>>
>>I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
>>stable series. ;)
> 
> 
> I thought a bit more about this issue and I have a different opinion now.
> 
> 2.6 is getting more and more stable and already includes libata --- users 
> who need it should use 2.6.

   While I agree that 2.6 is looking better every day.  I've been 
running my desktop on it for sometime.  I'm not sure agree we should be 
forcing  people to use 2.6 simply to be able to read their hard drive.


   It's getting harder to find a newly released motherboard without 
onboard sata.  Not having  libata support means that anyone running 2.4 
unpatched will be unable to use such systems.

-- 
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@rackable.com>

Re: libata in 2.4.24?

[Marcelo Tosatti]

On Sat, 29 Nov 2003, Marcelo Tosatti wrote:

> 
> 
> On Sat, 29 Nov 2003, Samuel Flory wrote:
> 
> >    Are you considering including libata support for 2.4.24?  From my 
> > testing with a number of different embedded sata chipsets (mostly ICH, 
> > SI, and Promise) it appears very stable.  I'm not seeing any data 
> > corruption or lockups when running Cerberus with 2.4.23-rc5 + libata 
> > patch.  The only troubles I've had were with initialization of embedded 
> > promise sata controllers. (I still need to test with Jeff's latest fixes 
> > for this.)
> 
> I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
> stable series. ;)

I thought a bit more about this issue and I have a different opinion now.

2.6 is getting more and more stable and already includes libata --- users 
who need it should use 2.6.

Re: libata in 2.4.24?

[Marcelo Tosatti]

On Sat, 29 Nov 2003, Samuel Flory wrote:

>    Are you considering including libata support for 2.4.24?  From my 
> testing with a number of different embedded sata chipsets (mostly ICH, 
> SI, and Promise) it appears very stable.  I'm not seeing any data 
> corruption or lockups when running Cerberus with 2.4.23-rc5 + libata 
> patch.  The only troubles I've had were with initialization of embedded 
> promise sata controllers. (I still need to test with Jeff's latest fixes 
> for this.)

I'm happy to include it in 2.4 when Jeff thinks its stable enough for a 
stable series. ;)

libata in 2.4.24?

[Samuel Flory]

   Are you considering including libata support for 2.4.24?  From my 
testing with a number of different embedded sata chipsets (mostly ICH, 
SI, and Promise) it appears very stable.  I'm not seeing any data 
corruption or lockups when running Cerberus with 2.4.23-rc5 + libata 
patch.  The only troubles I've had were with initialization of embedded 
promise sata controllers. (I still need to test with Jeff's latest fixes 
for this.)


-- 
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@rackable.com>