Ubuntu 11.04 Server Crashing - failed command: READ FPDMA QUEUED

I have a new Ubuntu Server (11.04) that keeps crashing, especially during heavy disk I/O (like making a backup). It's drives are configures as a RAID 10 with 4 1TB Western Digital Caviar Black Hard Drives.

The message I'm seeing via /proc/kmsg when it crashes is, "failed command: READ FPDMA QUEUED".

This seems like something is messed up with either the drives or the software raid is broken?

The machine was doing fine until this afternoon when it crashed during a file transfer Ever since then it's been crashing when I try to run a backup, but it's not always the same file or place each time.

How do I know if it's a software or hardware failure? How do I know if it's the SATA controller or one of the disks?

Also, all 4 drives in the array "completed without error" when I ran an extended off-line test on them.

This is the full output of /proc/kmsg from the time I rebooted until it crashed again:

[  356.076292] type=1400 audit(1311983491.536:14): apparmor="DENIED" operation="open" parent=1397 profile="/usr/lib/libvirt/virt-aa-helper" name="/dev/dm-9" pid=2222 comm="virt-aa-helper" requested_mask="r" denied_mask="r" fsuid=0 ouid=105
[  356.304840] type=1400 audit(1311983491.766:15): apparmor="STATUS" operation="profile_load" name="libvirt-c67f4a48-2cad-6deb-d7e7-13f9c7620ad9" pid=2223 comm="apparmor_parser"
[  357.002246] device vnet0 entered promiscuous mode
[  357.003702] br0: port 2(vnet0) entering learning state
[  357.003704] br0: port 2(vnet0) entering learning state
[  366.020017] br0: port 2(vnet0) entering forwarding state
[  367.050024] vnet0: no IPv6 routers present
 l)idx08f5
1[20.298 aefas x0000000(
4[20.294 i:38 om d_ad0Nttitd263-0sre 4-bnu 15054]Cl rc: 15055]  15055]  15055]  15055]  15055]  15056]   1bfpo_re05/x0[ad0
4[20.292 [ffff81de>  epo_re05/x0 15057]   ycrqetwie03b040[ad0
4[20.290 [ffffa017>  ad0+xe/x5 ri1] 15058]   eal_pnlc_lg+x/x0 15058]  15059]  15059]  15059]  15059]  15059]   enltra_epr0001
4[20.200 ialn okdbgigdet enltit_ x0000 H D hne
3[55.640 t4 Err  eoCm ess HRyh 08 
3[55.646 t40:fie omn:RA PM UUD 84088]aa.0 m 00:00:1d/40:90:04 a  c 228i
3[55.644     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.643 t40:sau:{DD 
3[55.647 t40:fie omn:RA PM UUD 84080]aa.0 m 00:80:dd/40:90:04 a  c 228i
3[55.654     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.653 t40:sau:{DD 
3[55.656 t40:fie omn:RA PM UUD 84082]aa.0 m 00:00:5d/40:90:04 a  c 228i
3[55.653     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.652 t40:sau:{DD 
3[55.656 t40:fie omn:RA PM UUD 84084]aa.0 m 00:80:9d/40:90:04 a  c 228i
3[55.652     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.651 t40:sau:{DD 
3[55.655 t40:fie omn:RA PM UUD 84086]aa.0 m 00:00:dd/40:90:04 a  c 228i
3[55.651     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.650 t40:sau:{DD 
3[55.654 t40:fie omn:RA PM UUD 84088]aa.0 m 00:80:1d/40:90:04 a  c 228i
3[55.650     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.650 t40:sau:{DD 
3[55.654 t40:fie omn:RA PM UUD 84089]aa.0 m 00:00:5d/40:90:04 a  c 228i
3[55.660     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.668 t40:sau:{DD 
3[55.662 t40:fie omn:RA PM UUD 84081]aa.0 m 00:80:9d/40:90:04 a  c 228i
3[55.668     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.668 t40:sau:{DD 
3[55.724 t40:fie omn:RA PM UUD:03:00/0tg8nq548 n 84008]     e 00:c0:9d/00:90:04 ms x0(T u ro) 84010]aa.0 tts  RY} 84029]aa.0 aldcmad EDFDAQEE
3[55.706 t40:cd6/04:01:a0:03:00/0tg9nq548 n 84034]     e 00:c0:9d/00:90:04 ms x0(T u ro) 84049]aa.0 tts  RY} 84048]aa.0 aldcmad EDFDAQEE
3[55.739 t40:cd6/05:01:a0:03:00/0tg1 c 228i
3[55.739     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.742 t40:sau:{DD 
3[55.703 t40:fie omn:RA PM UUD 84074]aa.0 m 00:80:9d/40:90:04 a 1nq548 n 84074]     e 00:c0:9d/00:90:04 ms x0(T u ro) 84083]aa.0 tts  RY} 84098]aa.0 aldcmad EDFDAQEE
3[55.797 t40:cd6/06:01:a0:03:00/0tg1 c 228i
3[55.798     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.811 t40:sau:{DD 
0 aldcmad EDFDAQEE
3[55.839 t40:cd6/06:0a:90:03:00/0tg1 c 228i
3[55.830     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.851 t40:sau:{DD 
3[55.818 t40:fie omn:RA PM UUD 84048]aa.0 m 00:00:9d/40:90:04 a 4nq548 n 84048]     e 00:c0:9d/00:90:04 ms x0(T u ro) 84058]aa.0 tts  RY} 84064]aa.0 aldcmad EDFDAQEE
3[55.809 t40:cd6/07:0a:90:03:00/0tg1 c 228i
3[55.800     rs4/05:01:a0:03:00/0Eak01 AAbserr
3[55.826 t40:sau:{DD 
3[55.871 t40:fie omn:RA PM UUD 84098]aa.0 m 00:00:1d/40:90:04 a 6nq548 n 84098]     e 00:c0:9d/00:90:04 ms x0(T u ro) 84007]aa.0 tts  RY}[ 5854.091012] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.091573] ata4.00: cmd 60/00:88:00:b5:d9/04:00:39:00:00/40 tag 17 ncq 524288 in
[ 5854.091574]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.092690] ata4.00: status: { DRDY }
[ 5854.093287] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.093817] ata4.00: cmd 60/00:90:00:b9:d9/04:00:39:00:00/40 tag 18 ncq 524288 in
[ 5854.093817]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.094964] ata4.00: status: { DRDY }
[ 5854.095510] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.096105] ata4.00: cmd 60/00:98:00:bd:d9/04:00:39:00:00/40 tag 19 ncq 524288 in
[ 5854.096105]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.097246] ata4.00: status: { DRDY }
[ 5854.097773] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.098338] ata4.00: cmd 60/00:a0:00:c1:d9/04:00:39:00:00/40 tag 20 ncq 524288 in
[ 5854.098339]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.099491] ata4.00: status: { DRDY }
[ 5854.100084] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.100658] ata4.00: cmd 60/00:a8:00:c5:d9/04:00:39:00:00/40 tag 21 ncq 524288 in
[ 5854.100659]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.101775] ata4.00: status: { DRDY }
[ 5854.102392] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.102924] ata4.00: cmd 60/00:b0:00:c9:d9/04:00:39:00:00/40 tag 22 ncq 524288 in
[ 5854.102925]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.104080] ata4.00: status: { DRDY }
[ 5854.104639] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.105206] ata4.00: cmd 60/00:b8:00:cd:d9/04:00:39:00:00/40 tag 23 ncq 524288 in
[ 5854.105207]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.106377] ata4.00: status: { DRDY }
[ 5854.106921] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.107493] ata4.00: cmd 60/00:c0:00:d1:d9/04:00:39:00:00/40 tag 24 ncq 524288 in
[ 5854.107493]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.108651] ata4.00: status: { DRDY }
[ 5854.109201] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.109784] ata4.00: cmd 60/00:c8:00:d5:d9/04:00:39:00:00/40 tag 25 ncq 524288 in
[ 5854.109785]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.110981] ata4.00: status: { DRDY }
[ 5854.111555] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.112111] ata4.00: cmd 60/00:d0:00:d9:d9/04:00:39:00:00/40 tag 26 ncq 524288 in
[ 5854.112111]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.113248] ata4.00: status: { DRDY }
[ 5854.113839] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.114408] ata4.00: cmd 60/00:d8:00:dd:d9/04:00:39:00:00/40 tag 27 ncq 524288 in
[ 5854.114409]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.115586] ata4.00: status: { DRDY }
[ 5854.116127] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.116691] ata4.00: cmd 60/00:e0:00:e1:d9/04:00:39:00:00/40 tag 28 ncq 524288 in
[ 5854.116691]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.117856] ata4.00: status: { DRDY }
[ 5854.118430] ata4.00: failed command: READ FPDMA QUEUED
[ 5854.119004] ata4.00: cmd 60/00:e8:00:e5:d9/04:00:39:00:00/40 tag 29 ncq 524288 in
[ 5854.119005]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
[ 5854.120213] ata4.00: status: { DRDY }
[ 5855.100038] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 5855.104300] ata4.00: configured for UDMA/133
[ 5855.104351] ata4: EH complete
[10013.907683] general protection fault: 0000 [#1] SMP 
[10013.907997] last sysfs file: /sys/devices/system/cpu/cpu5/cache/index2/shared_cpu_map
[10013.908574] CPU 0 
[10013.908577] Modules linked in: ip6table_filter ip6_tables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables kvm_amd kvm eeepc_wmi sparse_keymap bridge stp nouveau sp5100_tco ttm i2c_piix4 edac_core edac_mce_amd drm_kms_helper k10temp drm i2c_algo_bit video lp parport raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usb_storage uas r8169 xhci_hcd ahci libahci raid6_pq async_tx raid1 raid0 multipath linear
[10013.911080] 
[10013.911734] Pid: 349, comm: md0_resync Tainted: G    B       2.6.38-10-server #46-Ubuntu To be filled by O.E.M. To be filled by O.E.M./SABERTOOTH 990FX
[10013.912418] RIP: 0010:[]  [] kmem_cache_alloc+0x58/0x110
[10013.913102] RSP: 0018:ffff88041b6079c0  EFLAGS: 00010006
[10013.913783] RAX: 0000000000000000 RBX: ffff88043f802600 RCX: ffffffff813df60a
[10013.914472] RDX: 0000000000000000 RSI: 0000000000000020 RDI: ffff88043f802600
[10013.915159] RBP: ffff88041b607a00 R08: ffff8800bd416a80 R09: ffff880414db0500
[10013.915839] R10: 00000000684eb800 R11: 0000000000000001 R12: 0200000000000000
[10013.916523] R13: 0000000000000086 R14: 0000000000000020 R15: ffff88041b42b400
[10013.917204] FS:  00007f741ce31700(0000) GS:ffff8800bd400000(0000) knlGS:0000000000000000
[10013.917529] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[10013.917529] CR2: 00007f4402d5dd00 CR3: 00000004108a5000 CR4: 00000000000006f0
[10013.917529] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10013.917529] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10013.917529] Process md0_resync (pid: 349, threadinfo ffff88041b606000, task ffff88041b945b80)
[10013.917529] Stack:
[10013.917529]  ffff88041b6079e0 000000201b6079e0 0000000000000082 ffffffff81a77fa0
[10013.917529]  ffff880414db0500 0000000000000020 0000000000000020 ffff88041b42b400
[10013.917529]  ffff88041b607a30 ffffffff813df60a ffff880400000000 ffff88041be74020
[10013.917529] Call Trace:
[10013.917529]  [] scsi_pool_alloc_command+0x4a/0x80
[10013.917529]  [] scsi_host_alloc_command.clone.7+0x33/0xa0
[10013.159 [ffff83ff> _cigtcmad02/x0103972]  cistpf_md08/x0103972] 103972] 103972] 103972]  _eei_nlgdvc+x704
4[01.159 [ffff822c> eei_nlgdvc+x005
4[01.159 [ffff82c9> l_nlg02/x0103972]  ad0upu+xd03 ri1]103972]  dupu+x204
4[01.159 [ffff8421> dd_yc08d0c0103972] [10013.917529]  [] ? _raw_spin_lock+0xe/0x20
[10013.917529]  [] ? recalc_sigpending+0x1/x0103972] 103972] 103972] 103972] 103972]   enltra_epr0001
0[01.159 oe 66 06 04 9c a6 69 66 04 b0 54 30 5a c0 04 b2 d8 40 4a 00 04 34 8103972]RP 103972] S ff80167c>103972]-- n rc 5355716c]-

This is the output from syslog:

Jul 29 21:23:09 Othelo kernel: [ 5854.068449] ata4.00: exception Emask 0x10 SAct 0x7fffffff SErr 0x90202 action 0xe frozen
Jul 29 21:23:09 Othelo kernel: [ 5854.068464] ata4.00: irqstat00400,PYRYcagd<> 84087]aa:Sro:{RcvomPritPYdCg1BB}<> 84087]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.643 t40:cd6/00:0f:90:03:00/0tg0nq548 n<> 84088]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84089]aa.0 tts  RY}<> 84089]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.653 t40:cd6/00:0e:90:03:00/0tg1nq548 n<> 84080]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84081]aa.0 tts  RY}<> 84081]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.652 t40:cd6/01:0f:90:03:00/0tg2nq548 n<> 84082]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84083]aa.0 tts  RY}<> 84083]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.652 t40:cd6/01:0f:90:03:00/0tg3nq548 n<> 84084]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84085]aa.0 tts  RY}<> 84085]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.651 t40:cd6/02:0f:90:03:00/0tg4nq548 n<> 84086]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84087]aa.0 tts  RY}<> 84087]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.650 t40:cd6/02:00:a0:03:00/0tg5nq548 n<> 84088]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84089]aa.0 tts  RY}<> 84089]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.659 t40:cd6/03:00:a0:03:00/0tg6nq548 n<> 84080]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84080]aa.0 tts  RY}<> 84081]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.667 t40:cd6/03:00:a0:03:00/0tg7nq548 n<> 84081]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84099]aa.0 tts  RY}<> 84007]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: [ 5854.070785] ata4.00: cmd 60/00:40:00:0d:da/040:90:04 a  c 228i
Jul 29 21:23:09 Othelo kernel: 3[55.776     rs4/05:01:a0:03:00/0Eak01 AAbserr
Jul 29 21:23:09 Othelo kernel: 3[55.797 t40:sau:{DD 
Jul 29 21:23:09 Othelo kernel: 3[55.743 t40:fie omn:RA PM UUD<> 84034]aa.0 m 00:80:1d/40:90:04 a  c 228i
Jul 29 21:23:09 Othelo kernel: 3[55.707     rs4/05:01:a0:03:00/0Eak01 AAbserr
Jul 29 21:23:09 Othelo kernel: 3[55.710 t40:sau:{DD 
Jul 29 21:23:09 Othelo kernel: 3[55.777 t40:fie omn:RA PM UUD<> 84052]aa.0 m 00:00:5d/40:90:04 a 0nq548 n<> 84052]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84069]aa.0 tts  RY}<> 84075]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.763 t40:cd6/05:01:a0:03:00/0tg1 c 228i
Jul 29 21:23:09 Othelo kernel: 3[55.763     rs4/05:01:a0:03:00/0Eak01 AAbserr
Jul 29 21:23:09 Othelo kernel: 3[55.784 t40:sau:{DD 
Jul 29 21:23:09 Othelo kernel: 3[55.737 t40:fie omn:RA PM UUD<> 84096]aa.0 m 00:00:dd/40:90:04 a 2nq548 n<> 84096]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84018]aa.0 tts  RY}<3>[ 5854.081762] ata4.0:fie omn:RA PM UUD<> 84021]aa.0 m 00:80:5d/40:90:04 a 3nq548 n<> 84022]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84032]aa.0 tts  RY}<> 84045]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.867 t40:cd6/07:0a:90:03:00/0tg1 c 228i
Jul 29 21:23:09 Othelo kernel: 3[55.868     rs4/05:01:a0:03:00/0Eak01 AAbserr
Jul 29 21:23:09 Othelo kernel: 3[55.881 t40:sau:{DD 
Jul 29 21:23:09 Othelo kernel: 3[55.849 t40:fie omn:RA PM UUD<> 84073]aa.0 m 00:80:dd/40:90:04 a 5nq548 n<> 84074]     e 00:c0:9d/00:90:04 ms x0(T u ro)<> 84082]aa.0 tts  RY}<> 84083]aa.0 aldcmad EDFDAQEE
Jul 29 21:23:09 Othelo kernel: 3[55.822 t40:cd6/08:0b:90:03:00/0tg1 c 228i
Jul 29 21:23:09 Othelo kernel: 3[55.823     rs4/05:01:a0:03:00/0Eak01 AAbserr
Jul 29 21:23:09 Othelo kernel: 3[55.943 t40:sau:{DD 
Jul 29 21:23:09 Othelo kernel: [ 5854.120747] ata4.00: failed command: READ FPDMA QUEUED
Jul 29 21:23:09 Othelo kernel: [ 5854.121321] ata4.00: cmd 60/00:f0:00:e9:d9/04:00:39:00:00/40 tag 30 ncq 524288 in
Jul 29 21:23:09 Othelo kernel: [ 5854.121322]          res 40/00:5c:00:19:da/00:00:39:00:00/40 Emask 0x10 (ATA bus error)
Jul 29 21:23:09 Othelo kernel: [ 5854.122483] ata4.00: status: { DRDY }
Jul 29 21:23:09 Othelo kernel: [ 5854.123050] ata4: hard resetting link
Jul 29 21:42:45 Othelo mdadm[1563]: Rebuild62 event detected on md device /dev/md/0
Jul 29 22:16:06 Othelo mdadm[1563]: Rebuild81 event detected on md device /dev/md/0
Jul 29 22:17:01 Othelo CRON[13235]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jul 29 22:32:29 Othelo kernel: 4972] <ffff1d8e]_ss_e_omn+xe0c
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff83fd> cigtcmad04/x0<>103972] <ffff1efd]ss_eu_scn+xd0e
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff843a> dpe_n0a/xa
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff821b>  ediemv_eus+x80a
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff8222> l_ekrqet0c/x1
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff837c> cirqetf+x6040<>103972] <ffff1c77]_gnrcupu_eie03/x0<>103972] <ffff1ca0]gnrcupu_eie03/x0<>103972] <ffff1b75]bkupu+x506
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffffa0f8> nlgsae+x00c ri1]<>103972] <ffff0ead]ri1_nlg01/x0[ad0
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff82c9> l_nlg02/x0<>103972] <ffff19f2]m_nlg02/x0<>103972] <ffff198d]m_osn+x4/x9
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff807c>  uoeb05
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff84ca> dtra+x1/x5
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff84c9>  dtra+x/x5
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff8077> tra+x60a
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff80ce> enltra_epr0401
Jul 29 22:32:29 Othelo kernel: 4[01.159 [ffff807e>  tra+x/x0<>103972] <ffff10d0]?kre_hedhle+x/x0<>103972]Cd:6 69 69 98 5f 66 06 69 c8 76 c0 42 8d 00 d8 04 5e f8 00 00 86 71 4>8 40 98 04 9e 79 66 06 04 5e 5
Jul 29 22:32:29 Othelo kernel: 1[01.159 I [ffff815e> mmccealc05/x1
Jul 29 22:32:29 Othelo kernel: 4[01.159 RP<ff84b090

UPDATE: Checked all Hard drives by running full scan on each and writing zeros to all drives. No problems found. I'm going to reinstall everything and try again. :/

You answer your own question by mentioning changing a cable, zeroing the hard drives and starting from scratch seem to have fixed the problem, but to me it sounds like you could be having a case of the 550559 curse.

Therefore, I am wondering if you could comment the issue, whether it has returned or not, and if so how you fixed it the second time.

I don't have enough rep to comment on your answer, so that's why I'll provide this feedback in the form of an answer: Could be Bug #550559. 5 releases of Ubuntu later, still unassigned.

If so, tagging the bug with 'This bug affects you' would help make the bug more prone to be fixed.