linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* SATA hdd refuses to reallocate a sector?
@ 2013-06-23 10:19 Pavel Machek
  2013-06-23 11:21 ` Pavel Machek
  2013-06-24 12:48 ` Zdenek Kaspar
  0 siblings, 2 replies; 22+ messages in thread
From: Pavel Machek @ 2013-06-23 10:19 UTC (permalink / raw)
  To: kernel list, linux-ide

Hi!

This may very well be hw problem, but...

I have error on sda4. I tried to make hdd reallocate it by writing
zeros there, but it will not. Is there special kind of write that
needs to be done to force reallocation?

Would it be possible to indicate errors when writing to known-bad
sector?

root@amd:~# time cat /dev/zero > /dev/sda4
cat: write error: No space left on device

real 9m47.083s
user 0m0.264s
sys  1m24.424s
root@amd:~# time cat /dev/sda4  | wc -c
cat: /dev/sda4: Input/output error
8959361024

real	5m9.784s
user	0m0.544s
sys	2m11.620s
root@amd:~# time cat /dev/sda1  | wc -c
797852160

real	0m23.479s
user	0m0.040s
sys	0m5.492s

dmesg says:

...
wlan0: authenticated
iwl3945 0000:03:00.0 wlan0: disabling HT as WMM/QoS is not supported by the AP
iwl3945 0000:03:00.0 wlan0: disabling VHT as WMM/QoS is not supported by the AP
wlan0: associate with 00:11:95:05:30:d7 (try 1/3)
wlan0: RX AssocResp from 00:11:95:05:30:d7 (capab=0x401 status=0 aid=2)
wlan0: associated
ata1.00: exception Emask 0x0 SAct 0xf SErr 0x0 action 0x0
ata1.00: irq_stat 0x40000001
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/a8:00:b9:51:4b/00:00:39:00:00/40 tag 0 ncq 86016 in
         res 41/04:a8:c0:50:4b/00:00:39:00:00/00 Emask 0x1 (device error)
ata1.00: status: { DRDY ERR }
ata1.00: error: { ABRT }
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/58:08:61:52:4b/00:00:39:00:00/40 tag 1 ncq 45056 in
         res 41/04:a8:c0:50:4b/00:00:39:00:00/00 Emask 0x1 (device error)
ata1.00: status: { DRDY ERR }
ata1.00: error: { ABRT }
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/a8:10:b9:50:4b/00:00:39:00:00/40 tag 2 ncq 86016 in
         res 41/40:a8:c0:50:4b/00:00:39:00:00/00 Emask 0x409 (media error) <F>
ata1.00: status: { DRDY ERR }
ata1.00: error: { UNC }
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/58:18:61:51:4b/00:00:39:00:00/40 tag 3 ncq 45056 in
         res 41/04:a8:c0:50:4b/00:00:39:00:00/00 Emask 0x1 (device error)
ata1.00: status: { DRDY ERR }
ata1.00: error: { ABRT }
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Unhandled sense code
sd 0:0:0:0: [sda]  
Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda]  
Sense Key : 0x3 [current] [descriptor]
Descriptor sense data with sense descriptors (in hex):
        72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
        39 4b 50 c0 
sd 0:0:0:0: [sda]  
ASC=0x11 ASCQ=0x4
sd 0:0:0:0: [sda] CDB: 
cdb[0]=0x28: 28 00 39 4b 50 b9 00 00 a8 00
end_request: I/O error, dev sda, sector 961237184
Buffer I/O error on device sda4, logical block 17498759
Buffer I/O error on device sda4, logical block 17498760
Buffer I/O error on device sda4, logical block 17498761
Buffer I/O error on device sda4, logical block 17498762
Buffer I/O error on device sda4, logical block 17498763
Buffer I/O error on device sda4, logical block 17498764
Buffer I/O error on device sda4, logical block 17498765
Buffer I/O error on device sda4, logical block 17498766
Buffer I/O error on device sda4, logical block 17498767
Buffer I/O error on device sda4, logical block 17498768
ata1: EH complete
ata1.00: exception Emask 0x0 SAct 0xa SErr 0x0 action 0x0
ata1.00: irq_stat 0x40000008
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/01:18:c0:50:4b/00:00:39:00:00/40 tag 3 ncq 512 in
         res 41/40:01:c0:50:4b/00:00:39:00:00/00 Emask 0x409 (media error) <F>
ata1.00: status: { DRDY ERR }
ata1.00: error: { UNC }
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Unhandled sense code
sd 0:0:0:0: [sda]  
Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda]  
Sense Key : 0x3 [current] [descriptor]
Descriptor sense data with sense descriptors (in hex):
        72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
        39 4b 50 c0 
sd 0:0:0:0: [sda]  
ASC=0x11 ASCQ=0x4
sd 0:0:0:0: [sda] CDB: 
cdb[0]=0x28: 28 00 39 4b 50 c0 00 00 01 00
end_request: I/O error, dev sda, sector 961237184
ata1: EH complete



Disk seems mostly healthy otherwise, smart says:


smartctl 5.40 2010-07-12 r3124 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Momentus 5400.6 series
Device Model:     ST9500325AS
Serial Number:    5VE41HDA
Firmware Version: 0001SDM1
User Capacity:    500,107,862,016 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Jun 23 12:15:56 2013 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Self-test execution status:      ( 121)	The previous self-test completed having
					the read element of the test failed.
Total time to complete Offline 
data collection: 		 (   0) seconds.
Offline data collection
capabilities: 			 (0x73) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					No Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine 
recommended polling time: 	 (   1) minutes.
Extended self-test routine
recommended polling time: 	 ( 145) minutes.
Conveyance self-test routine
recommended polling time: 	 (   2) minutes.
SCT capabilities: 	       (0x103b)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   111   099   006    Pre-fail  Always       -       241452207
  3 Spin_Up_Time            0x0003   098   098   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   099   099   020    Old_age   Always       -       1036
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       3
  7 Seek_Error_Rate         0x000f   084   060   030    Pre-fail  Always       -       294188596
  9 Power_On_Hours          0x0032   068   068   000    Old_age   Always       -       28248
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   037   020    Old_age   Always       -       1023
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   074   074   000    Old_age   Always       -       26
188 Command_Timeout         0x0032   100   094   000    Old_age   Always       -       34360263770
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   052   045   045    Old_age   Always   In_the_past 48 (Lifetime Min/Max 20/55)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       1
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       1977103
194 Temperature_Celsius     0x0022   048   055   000    Old_age   Always       -       48 (0 6 0 0)
195 Hardware_ECC_Recovered  0x001a   051   029   000    Old_age   Always       -       241452207
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
254 Free_Fall_Sensor        0x0032   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
ATA Error Count: 24 (device log contains only the most recent five errors)
	CR = Command Register [HEX]
	FR = Features Register [HEX]
	SC = Sector Count Register [HEX]
	SN = Sector Number Register [HEX]
	CL = Cylinder Low Register [HEX]
	CH = Cylinder High Register [HEX]
	DH = Device/Head Register [HEX]
	DC = Device Command Register [HEX]
	ER = Error register [HEX]
	ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 24 occurred at disk power-on lifetime: 28248 hours (1177 days + 0 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 01 ff ff ff 4f 00  13d+01:29:41.468  READ FPDMA QUEUED
  60 00 a8 ff ff ff 4f 00  13d+01:29:41.468  READ FPDMA QUEUED
  60 00 58 ff ff ff 4f 00  13d+01:29:41.468  READ FPDMA QUEUED
  60 00 58 ff ff ff 4f 00  13d+01:29:41.467  READ FPDMA QUEUED
  27 00 00 00 00 00 e0 00  13d+01:29:41.465  READ NATIVE MAX ADDRESS EXT

Error 23 occurred at disk power-on lifetime: 28248 hours (1177 days + 0 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 58 ff ff ff 4f 00  13d+01:29:39.015  READ FPDMA QUEUED
  60 00 08 ff ff ff 4f 00  13d+01:29:38.699  READ FPDMA QUEUED
  60 00 a8 ff ff ff 4f 00  13d+01:29:38.699  READ FPDMA QUEUED
  60 00 a8 ff ff ff 4f 00  13d+01:29:38.699  READ FPDMA QUEUED
  60 00 a8 ff ff ff 4f 00  13d+01:29:38.698  READ FPDMA QUEUED

Error 22 occurred at disk power-on lifetime: 28248 hours (1177 days + 0 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 08 ff ff ff 4f 00  13d+01:08:08.641  READ FPDMA QUEUED
  27 00 00 00 00 00 e0 00  13d+01:08:08.641  READ NATIVE MAX ADDRESS EXT
  ec 00 00 00 00 00 a0 00  13d+01:08:08.639  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 00  13d+01:08:08.639  SET FEATURES [Set transfer mode]
  27 00 00 00 00 00 e0 00  13d+01:08:08.638  READ NATIVE MAX ADDRESS EXT

Error 21 occurred at disk power-on lifetime: 28248 hours (1177 days + 0 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 08 ff ff ff 4f 00  13d+01:08:06.101  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  13d+01:08:06.099  READ FPDMA QUEUED
  27 00 00 00 00 00 e0 00  13d+01:08:06.098  READ NATIVE MAX ADDRESS EXT
  ec 00 00 00 00 00 a0 00  13d+01:08:06.097  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 00  13d+01:08:06.096  SET FEATURES [Set transfer mode]

Error 20 occurred at disk power-on lifetime: 28248 hours (1177 days + 0 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 00 ff ff ff 4f 00  13d+01:07:55.327  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  13d+01:07:55.325  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  13d+01:07:55.324  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  13d+01:07:55.323  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  13d+01:07:55.320  READ FPDMA QUEUED

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed: read failure       90%     28233         961237188
# 2  Extended offline    Completed: read failure       90%     28227         961237188
# 3  Extended offline    Completed without error       00%     23826         -
# 4  Extended offline    Completed without error       00%      9419         -
# 5  Extended offline    Completed without error       00%        40         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

(Ok, this is slightly interesting:

 12 Power_Cycle_Count       0x0032   100   037   020    Old_age   Always       -       1023

AFAICT it indicates that power cycle count was too high but got
better. Ouch?)

Any ideas? Thanks,

									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 10:19 SATA hdd refuses to reallocate a sector? Pavel Machek
@ 2013-06-23 11:21 ` Pavel Machek
  2013-06-23 13:16   ` Marcus Overhagen
  2013-06-24 12:48 ` Zdenek Kaspar
  1 sibling, 1 reply; 22+ messages in thread
From: Pavel Machek @ 2013-06-23 11:21 UTC (permalink / raw)
  To: kernel list, linux-ide; +Cc: tj

On Sun 2013-06-23 12:19:40, Pavel Machek wrote:
> Hi!
> 
> This may very well be hw problem, but...
> 
> I have error on sda4. I tried to make hdd reallocate it by writing
> zeros there, but it will not. Is there special kind of write that
> needs to be done to force reallocation?
> 
> Would it be possible to indicate errors when writing to known-bad
> sector?

Uhuh. Seems like I have few consecutive bad sectors, and kernel is not
willing to overwrite them in one pass...?

root@amd:~# time cat /dev/zero > /dev/sda4
cat: write error: No space left on device

real 9m47.083s
user 0m0.264s
sys  1m24.424s
root@amd:~# time cat /dev/sda4  | wc -c
cat: /dev/sda4: Input/output error
8959361024

real	5m9.784s
user	0m0.544s
sys	2m11.620s
root@amd:~# time cat /dev/sda1  | wc -c
797852160

real	0m23.479s
user	0m0.040s
sys	0m5.492s
root@amd:~# time cat /dev/zero > /dev/sda4
cat: write error: No space left on device

real 7m51.619s
user 0m0.460s
sys  1m23.280s
root@amd:~# time cat /dev/sda4  | wc -c
cat: /dev/sda4: Input/output error
8958947328

real	5m25.973s
user	0m0.564s
sys	2m6.508s
root@amd:~# 

...Ok, lets try with dd.

root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=1 skip=8958947328
dd: reading `/dev/sda4': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 5.05805 s, 0.0 kB/s
root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=1 skip=8958947328
dd: reading `/dev/sda4': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 5.1051 s, 0.0 kB/s
root@amd:~# dd if=/dev/zero of=/dev/sda4 bs=1 seek=8958947328
dd: writing `/dev/sda4': Input/output error
3585+0 records in
3584+0 records out
3584 bytes (3.6 kB) copied, 2.63182 s, 1.4 kB/s
root@amd:~# 

...at least errors are propagated to dd. Aha, bad idea, I need to do
bigger block size so that I don't force reads...?

Better, but still not good:

root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
skip=$[8958947328/4096]
dd: reading `/dev/sda4': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 5.12378 s, 0.0 kB/s
root@amd:~# dd if=/dev/zero of=/dev/sda4 bs=4096
seek=$[8958947328/4096]
dd: writing `/dev/sda4': No space left on device
1746674+0 records in
1746673+0 records out
7154376192 bytes (7.2 GB) copied, 219.494 s, 32.6 MB/s
root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
skip=$[8958947328/4096]
dd: reading `/dev/sda4': Input/output error
101+0 records in
101+0 records out
413696 bytes (414 kB) copied, 4.94643 s, 83.6 kB/s
root@amd:~# 

Next try...

root@amd:~# dd if=/dev/zero of=/dev/sda4 bs=4096
seek=$[8958947328/4096]
dd: writing `/dev/sda4': No space left on device
1746674+0 records in
1746673+0 records out
7154376192 bytes (7.2 GB) copied, 241.291 s, 29.7 MB/s
root@amd:~# sync
root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
skip=$[8958947328/4096]
dd: reading `/dev/sda4': Input/output error
102+0 records in
102+0 records out
417792 bytes (418 kB) copied, 12.4968 s, 33.4 kB/s
root@amd:~# 

I don't think badblocks utility does what I need...?

I tried re-running dd few time, even with conf=fsync; but 

1) errors during write do not get reported

2) no more sectors are reallocated

root@amd:~# dd if=/dev/zero of=/dev/sda4 bs=4096
seek=$[8958947328/4096] conv=fsync
dd: writing `/dev/sda4': No space left on device
1746674+0 records in
1746673+0 records out
7154376192 bytes (7.2 GB) copied, 188.11 s, 38.0 MB/s
root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
skip=$[8958947328/4096]
dd: reading `/dev/sda4': Input/output error
102+0 records in
102+0 records out
417792 bytes (418 kB) copied, 6.20669 s, 67.3 kB/s
root@amd:~# 

Any ideas? Thanks,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 11:21 ` Pavel Machek
@ 2013-06-23 13:16   ` Marcus Overhagen
  2013-06-23 19:00     ` Pavel Machek
  0 siblings, 1 reply; 22+ messages in thread
From: Marcus Overhagen @ 2013-06-23 13:16 UTC (permalink / raw)
  To: Pavel Machek; +Cc: kernel list, linux-ide, tj

On Sun, Jun 23, 2013 at 1:21 PM, Pavel Machek <pavel@ucw.cz> wrote:

> ...Ok, lets try with dd.
>
> root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=1 skip=8958947328
> dd: reading `/dev/sda4': Input/output error
> 0+0 records in
> 0+0 records out
> 0 bytes (0 B) copied, 5.05805 s, 0.0 kB/s

I once noticed a similar problem. The trouble is that the kernel
always seems to be doing a larger read access that failes for this
sector, and the write is never executed.

You can try: hdparam --write-sector 961237188 /dev/sda

Hope that helps,

Marcus

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 13:16   ` Marcus Overhagen
@ 2013-06-23 19:00     ` Pavel Machek
  2013-06-23 21:27       ` Mark Lord
  0 siblings, 1 reply; 22+ messages in thread
From: Pavel Machek @ 2013-06-23 19:00 UTC (permalink / raw)
  To: Marcus Overhagen; +Cc: kernel list, linux-ide, tj, mlord

Hi!

> > ...Ok, lets try with dd.
> >
> > root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=1 skip=8958947328
> > dd: reading `/dev/sda4': Input/output error
> > 0+0 records in
> > 0+0 records out
> > 0 bytes (0 B) copied, 5.05805 s, 0.0 kB/s
> 
> I once noticed a similar problem. The trouble is that the kernel
> always seems to be doing a larger read access that failes for this
> sector, and the write is never executed.

And returns success writing? That's pretty antisocial :-(.

> You can try: hdparam --write-sector 961237188 /dev/sda

Thanks for the hint. (Insert rant about hdparm documentation
explaining that it is bad idea, but not telling me _why_ is it bad
idea. Can I expect cache consistency issues after that, or is it just
simple "you are writing to the disk without any checks"? Plus, I guess
documentation should mention what sector number is. I guess sectors
are 512bytes for the old drives, but is it 512 or 4096 for new
drives?)

...but it does not do the trick :-(. It behaves strangely as if it was
still cached somewhere. Do I need to turn off the write back cache?

root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
  read-sector: bad/missing sector value
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
  961237188 /dev/sda

/dev/sda:
reading sector 961237188: FAILED: Input/output error
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --write-sector
961237188 /dev/sda

/dev/sda:
re-writing sector 961237188: succeeded
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
reading sector 961237188: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
root@amd:~# sleep 10
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
reading sector 961237188: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
root@amd:~# sync
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
reading sector 961237188: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
skip=$[8958947328/4096]
dd: reading `/dev/sda4': Input/output error
102+0 records in
102+0 records out
417792 bytes (418 kB) copied, 5.36697 s, 77.8 kB/s
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
FAILED: Input/output error
reading sector 961237188: 
root@amd:~# 

Thanks,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 19:00     ` Pavel Machek
@ 2013-06-23 21:27       ` Mark Lord
  2013-06-23 21:51         ` Pavel Machek
  2015-04-30 19:01         ` Pavel Machek
  0 siblings, 2 replies; 22+ messages in thread
From: Mark Lord @ 2013-06-23 21:27 UTC (permalink / raw)
  To: Pavel Machek; +Cc: Marcus Overhagen, kernel list, linux-ide, tj

On 13-06-23 03:00 PM, Pavel Machek wrote:
>
> Thanks for the hint. (Insert rant about hdparm documentation
> explaining that it is bad idea, but not telling me _why_ is it bad
> idea. Can I expect cache consistency issues after that, or is it just
> simple "you are writing to the disk without any checks"? Plus, I guess
> documentation should mention what sector number is. I guess sectors
> are 512bytes for the old drives, but is it 512 or 4096 for new
> drives?)

For ATA, use the "logical sector size".
For all existing drives out there, that's a 512 byte unit.

> ...but it does not do the trick :-(. It behaves strangely as if it was
> still cached somewhere. Do I need to turn off the write back cache?

No, it works just fine.  You probably have more than one bad sector.
After you see a read failure, run "smartctl -a" and look at the error
logs to see what sector the drive is choking on.

Or just low-level format it all with "hdparm --security-erase".

Cheers
-- 
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 21:27       ` Mark Lord
@ 2013-06-23 21:51         ` Pavel Machek
  2013-06-23 22:35           ` Mark Lord
  2013-06-24  7:14           ` Ondrej Zary
  2015-04-30 19:01         ` Pavel Machek
  1 sibling, 2 replies; 22+ messages in thread
From: Pavel Machek @ 2013-06-23 21:51 UTC (permalink / raw)
  To: Mark Lord; +Cc: Marcus Overhagen, kernel list, linux-ide, tj

On Sun 2013-06-23 17:27:52, Mark Lord wrote:
> On 13-06-23 03:00 PM, Pavel Machek wrote:
> >
> > Thanks for the hint. (Insert rant about hdparm documentation
> > explaining that it is bad idea, but not telling me _why_ is it bad
> > idea. Can I expect cache consistency issues after that, or is it just
> > simple "you are writing to the disk without any checks"? Plus, I guess
> > documentation should mention what sector number is. I guess sectors
> > are 512bytes for the old drives, but is it 512 or 4096 for new
> > drives?)
> 
> For ATA, use the "logical sector size".
> For all existing drives out there, that's a 512 byte unit.

I guessed so. (It would be good to actually document it, as well as
documenting exactly why it is dangerous. Is it okay to send patches?)

> > ...but it does not do the trick :-(. It behaves strangely as if it was
> > still cached somewhere. Do I need to turn off the write back cache?
> 
> No, it works just fine.  You probably have more than one bad sector.
> After you see a read failure, run "smartctl -a" and look at the error
> logs to see what sector the drive is choking on.

Well, I definitely have more than one bad sector, but I did try to
read exactly the same sector and it failed. See below.

root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
 961237188 /dev/sda | uniq

/dev/sda:
FAILED: Input/output error
reading sector 961237188: 
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --write-sector
961237188 /dev/sda

/dev/sda:
re-writing sector 961237188: succeeded
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
FAILED: Input/output error
reading sector 961237188: 
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --write-sector
961237188 /dev/sda

/dev/sda:
re-writing sector 961237188: succeeded
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
reading sector 961237188: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
skip=$[8958947328/4096]
dd: reading `/dev/sda4': Input/output error
102+0 records in
102+0 records out
417792 bytes (418 kB) copied, 6.12536 s, 68.2 kB/s
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
reading sector 961237188: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
reading sector 961237188: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
961237188 /dev/sda | uniq

/dev/sda:
FAILED: Input/output error
reading sector 961237188: 
root@amd:~# 

> Or just low-level format it all with "hdparm --security-erase".

I'd like to understand what is going on there. I can mark the blocks
as bad at ext3 level, but I'd really like to understand what is going
on there, and if it is hw issue, sata issue or block layer issue.

(Plus, given that remapping does not work, I'd be afraid that it will
kill the disk for good).

The disk is

root@amd:~# smartctl -a /dev/sda
smartctl 5.40 2010-07-12 r3124 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen,
http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Momentus 5400.6 series
Device Model:     ST9500325AS
Serial Number:    5VE41HDA
Firmware Version: 0001SDM1
User Capacity:    500,107,862,016 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Jun 23 23:49:15 2013 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Thanks for support,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 21:51         ` Pavel Machek
@ 2013-06-23 22:35           ` Mark Lord
  2013-06-24  6:19             ` Marcus Overhagen
  2013-06-24  7:14           ` Ondrej Zary
  1 sibling, 1 reply; 22+ messages in thread
From: Mark Lord @ 2013-06-23 22:35 UTC (permalink / raw)
  To: Pavel Machek; +Cc: Marcus Overhagen, kernel list, linux-ide, tj

On 13-06-23 05:51 PM, Pavel Machek wrote:
> On Sun 2013-06-23 17:27:52, Mark Lord wrote:
>
>> For all existing drives out there, that's a 512 byte unit.
> 
> I guessed so. (It would be good to actually document it, as well as
> documenting exactly why it is dangerous. Is it okay to send patches?)

Absolutely.  Please, even!

> Well, I definitely have more than one bad sector, but I did try to
> read exactly the same sector and it failed. See below.
..
read failed.
write works.
read failed.
write works.
read works.
dd failed.
read works.
read works.
read failed.

Odd.  The drive must be furiously reshuffling sectors or something,
or more likely pushing a piece of dirt around scuffing up more bits.

hdparm generally talks directly to a drive, not through the block
or filesystem layers.  So the block, filesystem, and page-cache stuff
don't know anything about --read-sector and --write-sector.

Cheers
-- 
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 22:35           ` Mark Lord
@ 2013-06-24  6:19             ` Marcus Overhagen
  2013-06-24 12:28               ` Pavel Machek
  0 siblings, 1 reply; 22+ messages in thread
From: Marcus Overhagen @ 2013-06-24  6:19 UTC (permalink / raw)
  To: Mark Lord; +Cc: Pavel Machek, kernel list, linux-ide, tj

> > root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=1 skip=8958947328
> > > dd: reading `/dev/sda4': Input/output error
> > > 0+0 records in
> > > 0+0 records out
> > > 0 bytes (0 B) copied, 5.05805 s, 0.0 kB/s
> >
> > I once noticed a similar problem. The trouble is that the kernel
> > always seems to be doing a larger read access that failes for this
> > sector, and the write is never executed.
>
> And returns success writing? That's pretty antisocial :-(.

On a second look i see that you have if and of reversed in the dd command.
However, what I was referring to was that writing a single sector with dd
will fail with a io error because the kernel first seems to do a larger read.

> ...but it does not do the trick :-(. It behaves strangely as if it was
> still cached somewhere. Do I need to turn off the write back cache?

I used hdparm --write-sector successfully to fix a single sector where
dd would fail, but I really don't know what going on with your disk. I
guess your harddisk is having some more issues than this single
sector. If you haven't done it yet, make a complete backup!

Your original post contained:

> end_request: I/O error, dev sda, sector 961237184
> Buffer I/O error on device sda4, logical block 17498759
> Buffer I/O error on device sda4, logical block 17498760
> Buffer I/O error on device sda4, logical block 17498761
> Buffer I/O error on device sda4, logical block 17498762
> Buffer I/O error on device sda4, logical block 17498763
> Buffer I/O error on device sda4, logical block 17498764
> Buffer I/O error on device sda4, logical block 17498765
> Buffer I/O error on device sda4, logical block 17498766
> Buffer I/O error on device sda4, logical block 17498767
> Buffer I/O error on device sda4, logical block 17498768

So assuming(!) that sector 961237184of sda  is logical block 17498759
from sda4, you may need to write all sectors 961237184 to 961237193.

However, regarding the smart data, the drive still thinks that it's
pretty healthy, only 3 reallocated sectors, and no pending.

> 5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       3
> 197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
> 198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0

Perhaps writing the whole disk with dd and a larger blocksize would
ill work? something like

dd if=/dev/zero of=/dev/sda bs=1M

You shouldn't have any partiton monted when doing so. All data ist
lost after that is finished. then you can look into the smart data to
see how many sectors were reallocated, and decide if you want to trash
the disk.

regards
Marcus

On Mon, Jun 24, 2013 at 12:35 AM, Mark Lord <mlord@pobox.com> wrote:
> On 13-06-23 05:51 PM, Pavel Machek wrote:
>> On Sun 2013-06-23 17:27:52, Mark Lord wrote:
>>
>>> For all existing drives out there, that's a 512 byte unit.
>>
>> I guessed so. (It would be good to actually document it, as well as
>> documenting exactly why it is dangerous. Is it okay to send patches?)
>
> Absolutely.  Please, even!
>
>> Well, I definitely have more than one bad sector, but I did try to
>> read exactly the same sector and it failed. See below.
> ..
> read failed.
> write works.
> read failed.
> write works.
> read works.
> dd failed.
> read works.
> read works.
> read failed.
>
> Odd.  The drive must be furiously reshuffling sectors or something,
> or more likely pushing a piece of dirt around scuffing up more bits.
>
> hdparm generally talks directly to a drive, not through the block
> or filesystem layers.  So the block, filesystem, and page-cache stuff
> don't know anything about --read-sector and --write-sector.
>
> Cheers
> --
> Mark Lord
> Real-Time Remedies Inc.
> mlord@pobox.com

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 21:51         ` Pavel Machek
  2013-06-23 22:35           ` Mark Lord
@ 2013-06-24  7:14           ` Ondrej Zary
  2013-06-24 11:06             ` Pavel Machek
                               ` (2 more replies)
  1 sibling, 3 replies; 22+ messages in thread
From: Ondrej Zary @ 2013-06-24  7:14 UTC (permalink / raw)
  To: Pavel Machek; +Cc: Mark Lord, Marcus Overhagen, kernel list, linux-ide, tj

On Sunday 23 June 2013, Pavel Machek wrote:
> On Sun 2013-06-23 17:27:52, Mark Lord wrote:
> > On 13-06-23 03:00 PM, Pavel Machek wrote:
> > > Thanks for the hint. (Insert rant about hdparm documentation
> > > explaining that it is bad idea, but not telling me _why_ is it bad
> > > idea. Can I expect cache consistency issues after that, or is it just
> > > simple "you are writing to the disk without any checks"? Plus, I guess
> > > documentation should mention what sector number is. I guess sectors
> > > are 512bytes for the old drives, but is it 512 or 4096 for new
> > > drives?)
> >
> > For ATA, use the "logical sector size".
> > For all existing drives out there, that's a 512 byte unit.
>
> I guessed so. (It would be good to actually document it, as well as
> documenting exactly why it is dangerous. Is it okay to send patches?)
>
> > > ...but it does not do the trick :-(. It behaves strangely as if it was
> > > still cached somewhere. Do I need to turn off the write back cache?
> >
> > No, it works just fine.  You probably have more than one bad sector.
> > After you see a read failure, run "smartctl -a" and look at the error
> > logs to see what sector the drive is choking on.
>
> Well, I definitely have more than one bad sector, but I did try to
> read exactly the same sector and it failed. See below.
>
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
>  961237188 /dev/sda | uniq
>
> /dev/sda:
> FAILED: Input/output error
> reading sector 961237188:
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --write-sector
> 961237188 /dev/sda
>
> /dev/sda:
> re-writing sector 961237188: succeeded
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
> 961237188 /dev/sda | uniq
>
> /dev/sda:
> FAILED: Input/output error
> reading sector 961237188:
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --write-sector
> 961237188 /dev/sda
>
> /dev/sda:
> re-writing sector 961237188: succeeded
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
> 961237188 /dev/sda | uniq
>
> /dev/sda:
> reading sector 961237188: succeeded
> 0000 0000 0000 0000 0000 0000 0000 0000
> root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=4096
> skip=$[8958947328/4096]
> dd: reading `/dev/sda4': Input/output error
> 102+0 records in
> 102+0 records out
> 417792 bytes (418 kB) copied, 6.12536 s, 68.2 kB/s
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
> 961237188 /dev/sda | uniq
>
> /dev/sda:
> reading sector 961237188: succeeded
> 0000 0000 0000 0000 0000 0000 0000 0000
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
> 961237188 /dev/sda | uniq
>
> /dev/sda:
> reading sector 961237188: succeeded
> 0000 0000 0000 0000 0000 0000 0000 0000
> root@amd:~# hdparm --yes-i-know-what-i-am-doing  --read-sector
> 961237188 /dev/sda | uniq
>
> /dev/sda:
> FAILED: Input/output error
> reading sector 961237188:
> root@amd:~#
>
> > Or just low-level format it all with "hdparm --security-erase".
>
> I'd like to understand what is going on there. I can mark the blocks
> as bad at ext3 level, but I'd really like to understand what is going
> on there, and if it is hw issue, sata issue or block layer issue.
>
> (Plus, given that remapping does not work, I'd be afraid that it will
> kill the disk for good).
>
> The disk is
>
> root@amd:~# smartctl -a /dev/sda
> smartctl 5.40 2010-07-12 r3124 [i686-pc-linux-gnu] (local build)
> Copyright (C) 2002-10 by Bruce Allen,
> http://smartmontools.sourceforge.net
>
> === START OF INFORMATION SECTION ===
> Model Family:     Seagate Momentus 5400.6 series
> Device Model:     ST9500325AS
> Serial Number:    5VE41HDA
> Firmware Version: 0001SDM1
> User Capacity:    500,107,862,016 bytes
> Device is:        In smartctl database [for details use: -P show]
> ATA Version is:   8
> ATA Standard is:  ATA-8-ACS revision 4
> Local Time is:    Sun Jun 23 23:49:15 2013 CEST
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> Thanks for support,
> 									Pavel

Being tired of using hdparm manually, I created a simple hdd_realloc utility
that reads the disk in big blocks (1 MB). When there's a read error, it reads
the failed block sector-by-sector and tries to rewrite the sectors that fail
to read. It work fine for disks with just a couple of pending sectors.

#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>

#define BLOCK_SIZE	1048576
#define SECTOR_SIZE	512

int main(int argc, char *argv[]) {
	if (argc < 2) {
		fprintf(stderr, "Usage: %s <device> [pos]\n", argv[0]);
		return 1;
	}
	int dev = open(argv[1], O_RDWR | O_DIRECT | O_SYNC);
	if (dev < 1) {
		perror("Unable to open device");
		return 2;
	}

	posix_fadvise(dev, 0, 0, POSIX_FADV_RANDOM);

	off64_t startpos = 0, pos = 0;
	if (argc > 2) {
		sscanf(argv[2], "%lld", &startpos);
	}
	pos = startpos;
	char *buf = valloc(BLOCK_SIZE);
	char *zeros = valloc(SECTOR_SIZE);
	if (!buf || !zeros) {
		fprintf(stderr, "Memory allocation error\n");
		return 2;
	}
	memset(zeros, 0, SECTOR_SIZE);

	time_t starttime = time(NULL);

	while (1) {
		printf("\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b");
		printf("Position: %lld B (%lld MiB, %lld GiB, sector %lld), rate %lld MiB/s", pos, pos / 1024 / 1024,
			pos / 1024 / 1024 / 1024, pos / SECTOR_SIZE,
			(pos - startpos) / 1024 / 1024 / ((time(NULL) - starttime) ? (time(NULL) - starttime) : 1) );
		lseek64(dev, pos, SEEK_SET);
		int count = read(dev, buf, BLOCK_SIZE);
		if (count == 0)	{/* EOF */
			printf("End of disk\n");
			break;
		}
		if (count < 0) { /* read error */
			printf("\n");
			perror("Read error");
			printf("Examining %lld\n", pos);
			for (int i = 0; i < BLOCK_SIZE/SECTOR_SIZE; i++) {
				lseek64(dev, pos, SEEK_SET);
				if (read(dev, buf, SECTOR_SIZE) < SECTOR_SIZE) {
					printf("Unable to read at %lld, rewriting...", pos);
					lseek64(dev, pos, SEEK_SET);
					int result = write(dev, zeros, SECTOR_SIZE);
					if (result < 0) {
						printf("write error\n");
					} else {
						lseek64(dev, pos, SEEK_SET);
						if (read(dev, buf, SECTOR_SIZE) < SECTOR_SIZE)
							printf("read error after rewrite\n");
						else
							printf("OK\n");
					}
				}
				pos += SECTOR_SIZE;
			}
		} else /* no error */
			pos += count;
	}

	return 0;
}


-- 
Ondrej Zary

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-24  7:14           ` Ondrej Zary
@ 2013-06-24 11:06             ` Pavel Machek
  2013-06-24 12:18             ` Mark Lord
  2013-06-29 18:47             ` Henrique de Moraes Holschuh
  2 siblings, 0 replies; 22+ messages in thread
From: Pavel Machek @ 2013-06-24 11:06 UTC (permalink / raw)
  To: Ondrej Zary; +Cc: Mark Lord, Marcus Overhagen, kernel list, linux-ide, tj

Hi!

> > > For ATA, use the "logical sector size".
> > > For all existing drives out there, that's a 512 byte unit.
> >
> > I guessed so. (It would be good to actually document it, as well as
> > documenting exactly why it is dangerous. Is it okay to send patches?)
> >
> > > > ...but it does not do the trick :-(. It behaves strangely as if it was
> > > > still cached somewhere. Do I need to turn off the write back cache?
> > >
> > > No, it works just fine.  You probably have more than one bad sector.
> > > After you see a read failure, run "smartctl -a" and look at the error
> > > logs to see what sector the drive is choking on.
> >
> > Well, I definitely have more than one bad sector, but I did try to
> > read exactly the same sector and it failed. See below.
...
> > Thanks for support,
> > 									Pavel
> 
> Being tired of using hdparm manually, I created a simple hdd_realloc utility
> that reads the disk in big blocks (1 MB). When there's a read error, it reads
> the failed block sector-by-sector and tries to rewrite the sectors that fail
> to read. It work fine for disks with just a couple of pending
> sectors.

Thanks! This should really be part of distribution... And it does
O_DIRECT, O_SYNC...

It seems to have reallocated one more sector (I'm on 6 now).. but
there are still some underlying problems. I guess I should try stock
kernel from Debian?

Or perhaps give up and avoid Seagate in future :-(.

root@amd:/data/pavel/misc# ./hdd_realloc /dev/sda4 8958947328
Position: 8958947328 B (8543 MiB, 8 GiB, sector 17497944), rate 0
MiB/s
Read error: Input/output error
Examining 8958947328
Unable to read at 8959366144, rewriting...OK
Unable to read at 8959366656, rewriting...OK
Unable to read at 8959367168, rewriting...OK
Unable to read at 8959778816, rewriting...OK
Unable to read at 8959779840, rewriting...OK
Position: 8959995904 B (8544 MiB, 8 GiB, sector 17499992), rate 0
MiB/s
Read error: Input/output error
Examining 8959995904
Unable to read at 8960192512, rewriting...OK
Position: 12548222976 B (11966 MiB, 11 GiB^Csector 24506200), rate 10
MiB/s
root@amd:/data/pavel/misc# ./hdd_realloc /dev/sda4 8958947328
Position: 8958947328 B (8543 MiB, 8 GiB, sector 17497944), rate 0
MiB/s
Read error: Input/output error
Examining 8958947328
Unable to read at 8959366656, rewriting...OK
Unable to read at 8959367168, rewriting...OK
Position: 9^C6042624 B (9389 MiB, 9 GiB, sector 19230552), rate 11
MiB/s
root@amd:/data/pavel/misc# 

(s2ram to give disk a chance to reboot).

root@amd:/data/pavel/misc# ./hdd_realloc /dev/sda4 8958947328
Position: 8958947328 B (8543 MiB, 8 GiB, sector 17497944), rate 0
MiB/s
Read error: Input/output error
Examining 8958947328
Unable to read at 8959366656, rewriting...OK
Unable to read at 8959367168, rewriting...OK
Position: 14^C5529088 B (13928 MiB, 13 GiB, sector 28526424), rate 21
MiB/s
root@amd:/data/pavel/misc# ./hdd_realloc /dev/sda4 8958947328
Position: 8958947328 B (8543 MiB, 8 GiB, sector 17497944), rate 0
MiB/s
Read error: Input/output error
Examining 8958947328
Unable to read at 8959366656, rewriting...OK
Unable to read at 8959367168, rewriting...OK
Unable to read at 8959779840, rewriting...OK
Position: 9356357056 B (8921 MiB, 8 GiB, sector 18272088), rate 15
MiB/s

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-24  7:14           ` Ondrej Zary
  2013-06-24 11:06             ` Pavel Machek
@ 2013-06-24 12:18             ` Mark Lord
  2013-06-26  3:04               ` James Bottomley
  2013-06-29 18:47             ` Henrique de Moraes Holschuh
  2 siblings, 1 reply; 22+ messages in thread
From: Mark Lord @ 2013-06-24 12:18 UTC (permalink / raw)
  To: Ondrej Zary; +Cc: Pavel Machek, Marcus Overhagen, kernel list, linux-ide, tj

On 13-06-24 03:14 AM, Ondrej Zary wrote:
..
> Being tired of using hdparm manually, I created a simple hdd_realloc utility
> that reads the disk in big blocks (1 MB). When there's a read error, it reads
> the failed block sector-by-sector and tries to rewrite the sectors that fail
> to read. It work fine for disks with just a couple of pending sectors.

Something like that would work very well if it used the hdparm approach
(directly to the drive) for the sector-by-sector part.

Going through the block layer isn't always going to work,
because the kernel likes to do I/O in PAGE_SIZE multiples.

And the SCSI stack in Linux has rather atrocious error handling.
It lumps multiple requests together, and can fail the entire lot even
if only a single sector is bad.

Cheers
-- 
Mark Lord
Real-Time Remedies Inc.
mlord@pobox.com

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-24  6:19             ` Marcus Overhagen
@ 2013-06-24 12:28               ` Pavel Machek
  0 siblings, 0 replies; 22+ messages in thread
From: Pavel Machek @ 2013-06-24 12:28 UTC (permalink / raw)
  To: Marcus Overhagen; +Cc: Mark Lord, kernel list, linux-ide, tj

Hi!

> > > root@amd:~# dd if=/dev/sda4 of=/dev/zero bs=1 skip=8958947328
> > > > dd: reading `/dev/sda4': Input/output error
> > > > 0+0 records in
> > > > 0+0 records out
> > > > 0 bytes (0 B) copied, 5.05805 s, 0.0 kB/s
> > >
> > > I once noticed a similar problem. The trouble is that the kernel
> > > always seems to be doing a larger read access that failes for this
> > > sector, and the write is never executed.
> >
> > And returns success writing? That's pretty antisocial :-(.
> 
> On a second look i see that you have if and of reversed in the dd command.

Well, I was doing two dd commands, one to overwrite with zeros, one to
read...

> However, what I was referring to was that writing a single sector with dd
> will fail with a io error because the kernel first seems to do a
> larger read.

I see. But it seems that I'm not hitting this one.

> > ...but it does not do the trick :-(. It behaves strangely as if it was
> > still cached somewhere. Do I need to turn off the write back cache?
> 
> I used hdparm --write-sector successfully to fix a single sector where
> dd would fail, but I really don't know what going on with your disk. I
> guess your harddisk is having some more issues than this single
> sector. If you haven't done it yet, make a complete backup!

It looks like I went from ~10 bad sectors to ~2. But the last 2 seem
to be very persistent :-(.

> So assuming(!) that sector 961237184of sda  is logical block 17498759
> from sda4, you may need to write all sectors 961237184 to 961237193.
> 
> However, regarding the smart data, the drive still thinks that it's
> pretty healthy, only 3 reallocated sectors, and no pending.

Well, after remapping experiments, I'm up to 6 sectors reallocated,
and drive no longer thinks it is so healthy (see 187), but... still refuses.

  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       6
  7 Seek_Error_Rate         0x000f   084   060   030    Pre-fail  Always       -       294493007
  9 Power_On_Hours          0x0032   068   068   000    Old_age  Always       -       28275
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   099   037   020    Old_age  Always       -       1024
184 End-to-End_Error        0x0032   100   100   099    Old_age  Always       -       0
187 Reported_Uncorrect      0x0032   001   001   000    Old_age  Always       -       162

> Perhaps writing the whole disk with dd and a larger blocksize would
> ill work? something like
> 
> dd if=/dev/zero of=/dev/sda bs=1M
> 
> You shouldn't have any partiton monted when doing so. All data ist
> lost after that is finished. then you can look into the smart data to
> see how many sectors were reallocated, and decide if you want to trash
> the disk.

Well, the disk is 500GB, in notebook, and it is my primary
machine. I'm running backups now, but "full backup" is not exactly
easy, and doing complete erase would be few days of work :-(.

Fortunately, affected area is in small, ~15G partition, I don't really
_need_.

But I'd really like to understand what is going on there. New hdd for
the notebook is not out of question, but... AFAICT I can expect cca
4MB per second from the disk (it would be USB->SATA copy), and that
would be 34 hours...
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 10:19 SATA hdd refuses to reallocate a sector? Pavel Machek
  2013-06-23 11:21 ` Pavel Machek
@ 2013-06-24 12:48 ` Zdenek Kaspar
  2013-06-24 13:08   ` Ondrej Zary
  1 sibling, 1 reply; 22+ messages in thread
From: Zdenek Kaspar @ 2013-06-24 12:48 UTC (permalink / raw)
  To: linux-kernel; +Cc: linux-ide

On 06/23/2013 12:19 PM, Pavel Machek wrote:
> Hi!
> 
> This may very well be hw problem, but...
> 
> I have error on sda4. I tried to make hdd reallocate it by writing
> zeros there, but it will not. Is there special kind of write that
> needs to be done to force reallocation?

Hi Pavel, maybe smartctl -t long /dev/sda will do the trick reliable for
your drive..

Z.


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-24 12:48 ` Zdenek Kaspar
@ 2013-06-24 13:08   ` Ondrej Zary
  0 siblings, 0 replies; 22+ messages in thread
From: Ondrej Zary @ 2013-06-24 13:08 UTC (permalink / raw)
  To: Zdenek Kaspar; +Cc: linux-kernel, linux-ide

On Monday 24 June 2013, Zdenek Kaspar wrote:
> On 06/23/2013 12:19 PM, Pavel Machek wrote:
> > Hi!
> >
> > This may very well be hw problem, but...
> >
> > I have error on sda4. I tried to make hdd reallocate it by writing
> > zeros there, but it will not. Is there special kind of write that
> > needs to be done to force reallocation?
>
> Hi Pavel, maybe smartctl -t long /dev/sda will do the trick reliable for
> your drive..

It's a read-only test that ends with an error on the first bad sector.

-- 
Ondrej Zary

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-24 12:18             ` Mark Lord
@ 2013-06-26  3:04               ` James Bottomley
  2013-06-26  6:11                 ` Ondrej Zary
  0 siblings, 1 reply; 22+ messages in thread
From: James Bottomley @ 2013-06-26  3:04 UTC (permalink / raw)
  To: Mark Lord
  Cc: Ondrej Zary, Pavel Machek, Marcus Overhagen, kernel list, linux-ide, tj

On Mon, 2013-06-24 at 08:18 -0400, Mark Lord wrote:
> And the SCSI stack in Linux has rather atrocious error handling.
> It lumps multiple requests together, and can fail the entire lot even
> if only a single sector is bad.

That's rather misleading.  SCSI doesn't lump anything together; it
handles the requests it was passed.  For reads and writes through the
page cache, block will aggregate in the elevators, but you avoid that by
not using the page cache (O_DIRECT or SG_IO).

For devices which report failing sectors correctly data up to the failed
sector is returned and the request is shortened and retried from the
failed sector on.  If we get a second failure at the beginning (where
the previous bad sector was), then we give up.

James



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-26  3:04               ` James Bottomley
@ 2013-06-26  6:11                 ` Ondrej Zary
  0 siblings, 0 replies; 22+ messages in thread
From: Ondrej Zary @ 2013-06-26  6:11 UTC (permalink / raw)
  To: James Bottomley
  Cc: Mark Lord, Pavel Machek, Marcus Overhagen, kernel list, linux-ide, tj

On Wednesday 26 June 2013, James Bottomley wrote:
> On Mon, 2013-06-24 at 08:18 -0400, Mark Lord wrote:
> > And the SCSI stack in Linux has rather atrocious error handling.
> > It lumps multiple requests together, and can fail the entire lot even
> > if only a single sector is bad.
>
> That's rather misleading.  SCSI doesn't lump anything together; it
> handles the requests it was passed.  For reads and writes through the
> page cache, block will aggregate in the elevators, but you avoid that by
> not using the page cache (O_DIRECT or SG_IO).

Yes, it works fine with O_DIRECT - that's why hdd_realloc reads 
sector-by-sector when an error was detected. I'd also like to disable read 
retries but that does not seem to be possible.

> For devices which report failing sectors correctly data up to the failed
> sector is returned and the request is shortened and retried from the
> failed sector on.  If we get a second failure at the beginning (where
> the previous bad sector was), then we give up.
>
> James


-- 
Ondrej Zary

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-24  7:14           ` Ondrej Zary
  2013-06-24 11:06             ` Pavel Machek
  2013-06-24 12:18             ` Mark Lord
@ 2013-06-29 18:47             ` Henrique de Moraes Holschuh
  2013-06-29 23:02               ` Mark Lord
  2 siblings, 1 reply; 22+ messages in thread
From: Henrique de Moraes Holschuh @ 2013-06-29 18:47 UTC (permalink / raw)
  To: Ondrej Zary
  Cc: Pavel Machek, Mark Lord, Marcus Overhagen, kernel list, linux-ide, tj

You know, either the "long" or the "offline" SMART test routines do exactly
that on any spinning rust device with a firmware that is not utterly broken.

The HDD's firmware will rewrite, and even reallocate any "weak" sectors
found by the surface scan.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-29 18:47             ` Henrique de Moraes Holschuh
@ 2013-06-29 23:02               ` Mark Lord
  2013-06-30 14:34                 ` Henrique de Moraes Holschuh
  0 siblings, 1 reply; 22+ messages in thread
From: Mark Lord @ 2013-06-29 23:02 UTC (permalink / raw)
  To: Henrique de Moraes Holschuh
  Cc: Ondrej Zary, Pavel Machek, Mark Lord, Marcus Overhagen,
	kernel list, linux-ide, tj

On 13-06-29 02:47 PM, Henrique de Moraes Holschuh wrote:
> You know, either the "long" or the "offline" SMART test routines do exactly
> that on any spinning rust device with a firmware that is not utterly broken.
> 
> The HDD's firmware will rewrite, and even reallocate any "weak" sectors
> found by the surface scan.
> 

The drives I have tried this on (smartctl -t long),
abort at the first bad sector.  Not useful.


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-29 23:02               ` Mark Lord
@ 2013-06-30 14:34                 ` Henrique de Moraes Holschuh
  2013-06-30 16:49                   ` Pavel Machek
  0 siblings, 1 reply; 22+ messages in thread
From: Henrique de Moraes Holschuh @ 2013-06-30 14:34 UTC (permalink / raw)
  To: Mark Lord
  Cc: Ondrej Zary, Pavel Machek, Mark Lord, Marcus Overhagen,
	kernel list, linux-ide, tj

On Sat, 29 Jun 2013, Mark Lord wrote:
> On 13-06-29 02:47 PM, Henrique de Moraes Holschuh wrote:
> > You know, either the "long" or the "offline" SMART test routines do exactly
> > that on any spinning rust device with a firmware that is not utterly broken.
> > 
> > The HDD's firmware will rewrite, and even reallocate any "weak" sectors
> > found by the surface scan.
> > 
> 
> The drives I have tried this on (smartctl -t long),
> abort at the first bad sector.  Not useful.

Which vendor?  The Seagates I have on hand are not that crappy, for example
(their issues are subpar mechanics and the resulting high rate of failure,
but the firmware at least is not a piece of crap)...

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-30 14:34                 ` Henrique de Moraes Holschuh
@ 2013-06-30 16:49                   ` Pavel Machek
  2013-07-01 13:28                     ` Henrique de Moraes Holschuh
  0 siblings, 1 reply; 22+ messages in thread
From: Pavel Machek @ 2013-06-30 16:49 UTC (permalink / raw)
  To: Henrique de Moraes Holschuh
  Cc: Mark Lord, Ondrej Zary, Mark Lord, Marcus Overhagen, kernel list,
	linux-ide, tj

Hi!

> > > You know, either the "long" or the "offline" SMART test routines do exactly
> > > that on any spinning rust device with a firmware that is not utterly broken.
> > > 
> > > The HDD's firmware will rewrite, and even reallocate any "weak" sectors
> > > found by the surface scan.
> > > 
> > 
> > The drives I have tried this on (smartctl -t long),
> > abort at the first bad sector.  Not useful.
> 
> Which vendor?  The Seagates I have on hand are not that crappy, for example
> (their issues are subpar mechanics and the resulting high rate of failure,
> but the firmware at least is not a piece of crap)...

Are you sure? I seem to have firmware issues:

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Momentus 5400.6 series

... and yes, -t long terminates after first error :-(.
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-30 16:49                   ` Pavel Machek
@ 2013-07-01 13:28                     ` Henrique de Moraes Holschuh
  0 siblings, 0 replies; 22+ messages in thread
From: Henrique de Moraes Holschuh @ 2013-07-01 13:28 UTC (permalink / raw)
  To: Pavel Machek
  Cc: Mark Lord, Ondrej Zary, Mark Lord, Marcus Overhagen, kernel list,
	linux-ide, tj

On Sun, 30 Jun 2013, Pavel Machek wrote:
> > > > You know, either the "long" or the "offline" SMART test routines do exactly
> > > > that on any spinning rust device with a firmware that is not utterly broken.
> > > > 
> > > > The HDD's firmware will rewrite, and even reallocate any "weak" sectors
> > > > found by the surface scan.
> > > > 
> > > 
> > > The drives I have tried this on (smartctl -t long),
> > > abort at the first bad sector.  Not useful.
> > 
> > Which vendor?  The Seagates I have on hand are not that crappy, for example
> > (their issues are subpar mechanics and the resulting high rate of failure,
> > but the firmware at least is not a piece of crap)...
> 
> Are you sure? I seem to have firmware issues:
> 
> === START OF INFORMATION SECTION ===
> Model Family:     Seagate Momentus 5400.6 series

Apparently there is too much of a difference between Seagate Momentus and
Seagate Barracuda.  The barracuda are not crap [as far as the firmware
goes], where apparently the Momentus have crap firmware.

So we're both correct, and I was not precise enough when I spoke well of
Seagate firware :-)

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: SATA hdd refuses to reallocate a sector?
  2013-06-23 21:27       ` Mark Lord
  2013-06-23 21:51         ` Pavel Machek
@ 2015-04-30 19:01         ` Pavel Machek
  1 sibling, 0 replies; 22+ messages in thread
From: Pavel Machek @ 2015-04-30 19:01 UTC (permalink / raw)
  To: Mark Lord; +Cc: Marcus Overhagen, kernel list, linux-ide, tj

Hi!

> > Thanks for the hint. (Insert rant about hdparm documentation
> > explaining that it is bad idea, but not telling me _why_ is it bad
> > idea. Can I expect cache consistency issues after that, or is it just
> > simple "you are writing to the disk without any checks"? Plus, I guess
> > documentation should mention what sector number is. I guess sectors
> > are 512bytes for the old drives, but is it 512 or 4096 for new
> > drives?)
> 
> For ATA, use the "logical sector size".
> For all existing drives out there, that's a 512 byte unit.
> 
> > ...but it does not do the trick :-(. It behaves strangely as if it was
> > still cached somewhere. Do I need to turn off the write back cache?
> 
> No, it works just fine.  You probably have more than one bad sector.
> After you see a read failure, run "smartctl -a" and look at the error
> logs to see what sector the drive is choking on.
> 
> Or just low-level format it all with "hdparm --security-erase".

Ok, so security erase is not easy operation to do, and it took more
than two hours, and the drive is now full of zeros.. but the bad
sectors are still there.

smartctl -t long stopped working on this one. It is no longer possible
to start a test.

It seems the drive has some firmware problems... But smart still
indicates good health and it still works (except few bad sectors).

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2015-04-30 19:01 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-06-23 10:19 SATA hdd refuses to reallocate a sector? Pavel Machek
2013-06-23 11:21 ` Pavel Machek
2013-06-23 13:16   ` Marcus Overhagen
2013-06-23 19:00     ` Pavel Machek
2013-06-23 21:27       ` Mark Lord
2013-06-23 21:51         ` Pavel Machek
2013-06-23 22:35           ` Mark Lord
2013-06-24  6:19             ` Marcus Overhagen
2013-06-24 12:28               ` Pavel Machek
2013-06-24  7:14           ` Ondrej Zary
2013-06-24 11:06             ` Pavel Machek
2013-06-24 12:18             ` Mark Lord
2013-06-26  3:04               ` James Bottomley
2013-06-26  6:11                 ` Ondrej Zary
2013-06-29 18:47             ` Henrique de Moraes Holschuh
2013-06-29 23:02               ` Mark Lord
2013-06-30 14:34                 ` Henrique de Moraes Holschuh
2013-06-30 16:49                   ` Pavel Machek
2013-07-01 13:28                     ` Henrique de Moraes Holschuh
2015-04-30 19:01         ` Pavel Machek
2013-06-24 12:48 ` Zdenek Kaspar
2013-06-24 13:08   ` Ondrej Zary

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).