Testing faulty disk
I have a degraded RAID-1 setup and I'm trying to troubleshoot the problem whether it's completely dead (and need to send the drive back to the vendor) or it can be brought back.
So I ran smartctl:
root@linux:~# smartctl -l selftest /dev/sda
smartctl 6.5 2016-01-24 r4214 [x86_64-linux-4.4.0-176-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 1930 -
# 2 Short offline Completed without error 00% 1930 -
# 3 Extended offline Completed without error 00% 1930 -
Looks good. Let's take a look at the other stuff:
root@linux:~# smartctl --all /dev/sda
smartctl 6.5 2016-01-24 r4214 [x86_64-linux-4.4.0-176-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red
Device Model: WDC WD40EFRX-68N32N0
Serial Number: WD-WCC7K3NU4V6D
LU WWN Device Id: 5 0014ee 211e15108
Firmware Version: 82.00A82
User Capacity: 4,000,787,030,016 bytes [4.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Mar 29 23:04:32 2020 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (44340) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 470) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x303d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 100 253 021 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 3
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 098 098 000 Old_age Always - 1930
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 3
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 0
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 1286
194 Temperature_Celsius 0x0022 117 114 000 Old_age Always - 33
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0
SMART Error Log Version: 1
ATA Error Count: 1757 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 1757 occurred at disk power-on lifetime: 1930 hours (80 days + 10 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 02 00 00 00 a0 Device Fault; Error: ABRT
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 10 02 00 00 00 a0 08 11d+12:18:30.077 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.077 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 08 11d+12:18:30.077 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 08 11d+12:18:30.073 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.073 IDENTIFY DEVICE
Error 1756 occurred at disk power-on lifetime: 1930 hours (80 days + 10 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 46 00 00 00 a0 Device Fault; Error: ABRT
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 03 46 00 00 00 a0 08 11d+12:18:30.077 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 08 11d+12:18:30.073 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.073 IDENTIFY DEVICE
c8 00 08 08 00 00 e0 08 11d+12:18:30.051 READ DMA
ef 10 02 00 00 00 a0 08 11d+12:18:30.041 SET FEATURES [Enable SATA feature]
Error 1755 occurred at disk power-on lifetime: 1930 hours (80 days + 10 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 02 00 00 00 a0 Device Fault; Error: ABRT
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 10 02 00 00 00 a0 08 11d+12:18:30.073 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.073 IDENTIFY DEVICE
c8 00 08 08 00 00 e0 08 11d+12:18:30.051 READ DMA
ef 10 02 00 00 00 a0 08 11d+12:18:30.041 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.040 IDENTIFY DEVICE
Error 1754 occurred at disk power-on lifetime: 1930 hours (80 days + 10 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 08 08 00 00 e0 Device Fault; Error: ABRT 8 sectors at LBA = 0x00000008 = 8
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
c8 00 08 08 00 00 e0 08 11d+12:18:30.051 READ DMA
ef 10 02 00 00 00 a0 08 11d+12:18:30.041 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.040 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 08 11d+12:18:30.040 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 08 11d+12:18:30.037 SET FEATURES [Enable SATA feature]
Error 1753 occurred at disk power-on lifetime: 1930 hours (80 days + 10 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 02 00 00 00 a0 Device Fault; Error: ABRT
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 10 02 00 00 00 a0 08 11d+12:18:30.041 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.040 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 08 11d+12:18:30.040 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 08 11d+12:18:30.037 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 08 11d+12:18:30.037 IDENTIFY DEVICE
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 1930 -
# 2 Short offline Completed without error 00% 1930 -
# 3 Extended offline Completed without error 00% 1930 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
I tried to look through the output but unfortunately I'm not an expert in analyzing smartctl output. It said the drive passed the tests:
SMART overall-health self-assessment test result: PASSED
In the tresholds section everything looks okay (this is a rather new WD Red HDD, a few months old).
At the end of the output there are tons of errors though. I tried to look them up with no luck.
On the other hand, I cannot use the disk at all:
root@idealib:~# fdisk /dev/sda
Welcome to fdisk (util-linux 2.27.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
fdisk: cannot open /dev/sda: Input/output error
dmesg looks bad as well:
[ 1708.769491] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
[ 1708.771763] ata1.00: irq_stat 0x40000001
[ 1708.773710] ata1.00: failed command: READ DMA
[ 1708.775667] ata1.00: cmd c8/00:08:08:00:00/00:00:00:00:00/e0 tag 15 dma 4096 in
res 61/04:08:08:00:00/00:00:00:00:00/e0 Emask 0x1 (device error)
[ 1708.779619] ata1.00: status: { DRDY DF ERR }
[ 1708.781613] ata1.00: error: { ABRT }
[ 1708.784344] ata1.00: failed to enable AA (error_mask=0x1)
[ 1708.787355] ata1.00: failed to enable AA (error_mask=0x1)
[ 1708.789183] ata1.00: configured for UDMA/133 (device error ignored)
[ 1708.796930] ata1: EH complete
What other test should I run before I can be 100% sure that the disk is faulty and I need replace it? Or it's dead and I should just accept it? :)
asked on Server Fault Mar 29, 2020 by 
riska
riska1 Answer
I can't help with the SMART data (I'm not that smart!) but my default tool to poke at the drive is badblocks.
Is a tool for finding bad blocks on a hard drive. If you're okay deleting data, it's worth running it in destructive mode - note this will take some time to run!:
badblocks -wsv /dev/sda
There's actually pretty good information on the Arch wiki (https://wiki.archlinux.org/index.php/badblocks) about the tool and how to use it.
Another tool you could try is Spinrite, which operates from a freedos environment at boot. It's a little old, but does go quite low-level on the disk. I'd suggest a level 4 scan if you use it. https://www.grc.com/sr/spinrite.htm
I'm sure you've already done so, but I always suggest checking/changing the SATA cable, port on the motherboard, etc. I've been caught out by this kind of problem more times than I care to admit.
answered on Server Fault Mar 29, 2020 by 
SamR.
SamR.User contributions licensed under CC BY-SA 3.0