Re: Drive tester

Re: Drive tester

[Greg Freemyer]

cc'ing: linux-ide@vger.kernel.org

maybe they can help

On Thu, Jul 23, 2009 at 6:12 PM, Bill Weiler<weilerb@marvell.com> wrote:
>
>
> ________________________________________
> From: Greg Freemyer [greg.freemyer@gmail.com]
> Sent: Thursday, July 23, 2009 2:31 PM
> To: Bill Weiler
> Cc: Kernelnewbies
> Subject: Re: Drive tester
>
> On Wed, Jul 22, 2009 at 7:55 PM, Bill Weiler<weilerb@marvell.com> wrote:
>> I wanted to make an Application that could send scripted SCSI and SATA
>> commands to my drives. I have found my Fedora Linux to be too complex to
>> control the drives in this way. I have looked at minimal Linux but I think
>> this would have the same problem. Is there an easier Open Source OS, or an
>> embedded PC Linux that would be easier to modify?
>
> Bill,
>
> For sata drives, have you looked at the SG_IO interface?
>
> That is how hdparm interfaces with the kernel.  I know it sends a lot
> of low level ATA commands to the drive from user space.  I think most
> use the SG_IO interface.  How much simpler can it get?
>
> strace hdparm may give you an education as to how simple it is.
>
> fyi: I believe the kernel looks for dangerous ata commands and blocks
> them.  But I think those are the types of commands that cause data
> loss.  Not sure what the criteria is.
>
> I have studied hdparm and the SG_IO interface and delved into the scsi drivers and ata drivers.
>
> For example, I wanted to script a single NCQ and a single NCQ write. A normal read(fd,buf,4096) does 1 NCQ read but a write(fd,buf,4096) does dozens of NCQ reads before doing the 1 NCQ write. How do I get rid of these reads?
>
> Also, I need to do soft and hard resets and these are not exposed.

Re: Drive tester

[Robert Hancock]

On 07/23/2009 04:31 PM, Greg Freemyer wrote:
> cc'ing: linux-ide@vger.kernel.org
>
> maybe they can help
>
> On Thu, Jul 23, 2009 at 6:12 PM, Bill Weiler<weilerb@marvell.com>  wrote:
>>
>> ________________________________________
>> From: Greg Freemyer [greg.freemyer@gmail.com]
>> Sent: Thursday, July 23, 2009 2:31 PM
>> To: Bill Weiler
>> Cc: Kernelnewbies
>> Subject: Re: Drive tester
>>
>> On Wed, Jul 22, 2009 at 7:55 PM, Bill Weiler<weilerb@marvell.com>  wrote:
>>> I wanted to make an Application that could send scripted SCSI and SATA
>>> commands to my drives. I have found my Fedora Linux to be too complex to
>>> control the drives in this way. I have looked at minimal Linux but I think
>>> this would have the same problem. Is there an easier Open Source OS, or an
>>> embedded PC Linux that would be easier to modify?
>> Bill,
>>
>> For sata drives, have you looked at the SG_IO interface?
>>
>> That is how hdparm interfaces with the kernel.  I know it sends a lot
>> of low level ATA commands to the drive from user space.  I think most
>> use the SG_IO interface.  How much simpler can it get?
>>
>> strace hdparm may give you an education as to how simple it is.

The best way to issue arbitrary ATA commands is likely to issue them as 
ATA pass-through SCSI commands using SG_IO, yes.

>>
>> fyi: I believe the kernel looks for dangerous ata commands and blocks
>> them.  But I think those are the types of commands that cause data
>> loss.  Not sure what the criteria is.

If you're running as unprivileged user I believe we block things like 
could do things like brick the drive such as uploading firmware, etc. 
However, if you have read-write access to the device node you can still 
read and write sectors anywhere on the disk. A few commands are blocked 
even for root users, like SET FEATURES - XFER MODE as it will screw up 
the mode configuration, but it's mostly fair game.

>>
>> I have studied hdparm and the SG_IO interface and delved into the scsi drivers and ata drivers.
>>
>> For example, I wanted to script a single NCQ and a single NCQ write. A normal read(fd,buf,4096) does 1 NCQ read but a write(fd,buf,4096) does dozens of NCQ reads before doing the 1 NCQ write. How do I get rid of these reads?

Try opening the device node with O_DIRECT. That will prevent pagecache 
effects from changing the IO pattern. However, you can't get less than 
page-size (4K granularity) on requests, for that you'll need to issue 
SCSI or ATA commands directly.

>>
>> Also, I need to do soft and hard resets and these are not exposed.

Indeed, there's no way currently to trigger these other than as part of 
error handling (i.e. command errors, hotplug operations, etc.) There's 
likely no reason it couldn't be added, but obviously nobody's needed it 
enough to bother so far..

Re: Drive tester

[Greg Freemyer]

On Fri, Jul 24, 2009 at 1:34 AM, Robert Hancock<hancockrwd@gmail.com> wrote:
> On 07/23/2009 04:31 PM, Greg Freemyer wrote:
>>
>> cc'ing: linux-ide@vger.kernel.org
>>
>> maybe they can help
>>
>> On Thu, Jul 23, 2009 at 6:12 PM, Bill Weiler<weilerb@marvell.com>  wrote:
>>>
>>> ________________________________________
>>> From: Greg Freemyer [greg.freemyer@gmail.com]
>>> Sent: Thursday, July 23, 2009 2:31 PM
>>> To: Bill Weiler
>>> Cc: Kernelnewbies
>>> Subject: Re: Drive tester
>>>
>>> On Wed, Jul 22, 2009 at 7:55 PM, Bill Weiler<weilerb@marvell.com>  wrote:
>>>>
>>>> I wanted to make an Application that could send scripted SCSI and SATA
>>>> commands to my drives. I have found my Fedora Linux to be too complex to
>>>> control the drives in this way. I have looked at minimal Linux but I
>>>> think
>>>> this would have the same problem. Is there an easier Open Source OS, or
>>>> an
>>>> embedded PC Linux that would be easier to modify?
>>>
>>> Bill,
>>>
>>> For sata drives, have you looked at the SG_IO interface?
>>>
>>> That is how hdparm interfaces with the kernel.  I know it sends a lot
>>> of low level ATA commands to the drive from user space.  I think most
>>> use the SG_IO interface.  How much simpler can it get?
>>>
>>> strace hdparm may give you an education as to how simple it is.
>
> The best way to issue arbitrary ATA commands is likely to issue them as ATA
> pass-through SCSI commands using SG_IO, yes.
>
>>>
>>> fyi: I believe the kernel looks for dangerous ata commands and blocks
>>> them.  But I think those are the types of commands that cause data
>>> loss.  Not sure what the criteria is.
>
> If you're running as unprivileged user I believe we block things like could
> do things like brick the drive such as uploading firmware, etc. However, if
> you have read-write access to the device node you can still read and write
> sectors anywhere on the disk. A few commands are blocked even for root
> users, like SET FEATURES - XFER MODE as it will screw up the mode
> configuration, but it's mostly fair game.
>
>>>
>>> I have studied hdparm and the SG_IO interface and delved into the scsi
>>> drivers and ata drivers.
>>>
>>> For example, I wanted to script a single NCQ and a single NCQ write. A
>>> normal read(fd,buf,4096) does 1 NCQ read but a write(fd,buf,4096) does
>>> dozens of NCQ reads before doing the 1 NCQ write. How do I get rid of these
>>> reads?
>
> Try opening the device node with O_DIRECT. That will prevent pagecache
> effects from changing the IO pattern. However, you can't get less than
> page-size (4K granularity) on requests, for that you'll need to issue SCSI
> or ATA commands directly.
>
>>>
>>> Also, I need to do soft and hard resets and these are not exposed.
>
> Indeed, there's no way currently to trigger these other than as part of
> error handling (i.e. command errors, hotplug operations, etc.) There's
> likely no reason it couldn't be added, but obviously nobody's needed it
> enough to bother so far..
>

Bill,

At this point are you only having issues with hard and soft resets?

If so, maybe Mark Lord could be talked into extending hdparm and the
kernel to support those.  Then you could leverage his work in your own
user space code.

Greg
-- 
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
Preservation and Forensic processing of Exchange Repositories White Paper -
<http://www.norcrossgroup.com/forms/whitepapers/tng_whitepaper_fpe.html>

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com

Re: Drive tester

[Gwendal Grignou]

On Thu, Jul 23, 2009 at 10:34 PM, Robert Hancock<hancockrwd@gmail.com> wrote:
> On 07/23/2009 04:31 PM, Greg Freemyer wrote:
>>
>> cc'ing: linux-ide@vger.kernel.org
>>
>> maybe they can help
>>
>> On Thu, Jul 23, 2009 at 6:12 PM, Bill Weiler<weilerb@marvell.com>  wrote:
>>>
>>> ________________________________________
>>> From: Greg Freemyer [greg.freemyer@gmail.com]
>>> Sent: Thursday, July 23, 2009 2:31 PM
>>> To: Bill Weiler
>>> Cc: Kernelnewbies
>>> Subject: Re: Drive tester
>>>
>>> On Wed, Jul 22, 2009 at 7:55 PM, Bill Weiler<weilerb@marvell.com>  wrote:
>>>>
>>>> I wanted to make an Application that could send scripted SCSI and SATA
>>>> commands to my drives. I have found my Fedora Linux to be too complex to
>>>> control the drives in this way. I have looked at minimal Linux but I
>>>> think
>>>> this would have the same problem. Is there an easier Open Source OS, or
>>>> an
>>>> embedded PC Linux that would be easier to modify?
>>>
>>> Bill,
>>>
>>> For sata drives, have you looked at the SG_IO interface?
>>>
>>> That is how hdparm interfaces with the kernel.  I know it sends a lot
>>> of low level ATA commands to the drive from user space.  I think most
>>> use the SG_IO interface.  How much simpler can it get?
>>>
>>> strace hdparm may give you an education as to how simple it is.
>
> The best way to issue arbitrary ATA commands is likely to issue them as ATA
> pass-through SCSI commands using SG_IO, yes.
>
>>>
>>> fyi: I believe the kernel looks for dangerous ata commands and blocks
>>> them.  But I think those are the types of commands that cause data
>>> loss.  Not sure what the criteria is.
>
> If you're running as unprivileged user I believe we block things like could
> do things like brick the drive such as uploading firmware, etc. However, if
> you have read-write access to the device node you can still read and write
> sectors anywhere on the disk. A few commands are blocked even for root
> users, like SET FEATURES - XFER MODE as it will screw up the mode
> configuration, but it's mostly fair game.
>
>>>
>>> I have studied hdparm and the SG_IO interface and delved into the scsi
>>> drivers and ata drivers.
>>>
>>> For example, I wanted to script a single NCQ and a single NCQ write. A
>>> normal read(fd,buf,4096) does 1 NCQ read but a write(fd,buf,4096) does
>>> dozens of NCQ reads before doing the 1 NCQ write. How do I get rid of these
>>> reads?
>
> Try opening the device node with O_DIRECT. That will prevent pagecache
> effects from changing the IO pattern. However, you can't get less than
> page-size (4K granularity) on requests, for that you'll need to issue SCSI
> or ATA commands directly.
>
>>>
>>> Also, I need to do soft and hard resets and these are not exposed.
>
> Indeed, there's no way currently to trigger these other than as part of
> error handling (i.e. command errors, hotplug operations, etc.) There's
> likely no reason it couldn't be added, but obviously nobody's needed it
> enough to bother so far..

We can trigger reset from user space using scsi scan interface.
hard/soft is decided by the interface driver, in particular if it has
access to SATA SCR registers or is working in IDE backward
compatibility mode.
Once you know the scsi location { host_id, channel_id, target_id,
lun_id } of you ATA disk, issue

echo <channel_id> <target_id> <lun_id> > /sys/class/scsi_host/host<host_id>/scan

For instance, on my system:
echo 0 0 0 > /sys/class/scsi_host/host5/scan

Will generate:
ata6: hard resetting link
ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata6.00: configured for UDMA/133
ata6: EH complete

> --
> To unsubscribe from this list: send the line "unsubscribe linux-ide" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>