Ubuntu Server 18.04 - hardware RAID Card will always be offline and RreadOnly

-1

I am gotting this on my server:

system: Ubuntu 18.04.2 LTS
RAID Card: RAID 5, LSI Logic / Symbios Logic MegaRAID Tri-Mode SAS3508
CPU: AMD 3950X
GPU: NVIDIA 1080Ti
RAM: 256G
DISK: 10T SAS * 4

The energy is ok, with a power supply of 1250W.

And here is the problem: RAID will come offline on some machine, and it will become to ro mode, which is unrecoverable unless reboot.

/dev/sda1 on /data type ext4 (ro,relatime,stripe=192,data=ordered)

The good mount is: /dev/sda1 on /data type ext4 (rw,relatime,stripe=192,data=ordered)

We have around 33TiB to write to disk per day(write and move to somewhere else and write again), not so sure which has caused this problem.

Syslog here:

Sep 18 19:37:59 AMD-12 kernel: [380860.166651] INFO: task jbd2/sda1-8:1433 blocked for more than 120 seconds.
Sep 18 19:37:59 AMD-12 kernel: [380860.166659]       Tainted: P        W  OE    4.15.0-45-generic #48-Ubuntu
Sep 18 19:37:59 AMD-12 kernel: [380860.166662] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 18 19:37:59 AMD-12 kernel: [380860.166665] jbd2/sda1-8     D    0  1433      2 0x80000000
Sep 18 19:37:59 AMD-12 kernel: [380860.166668] Call Trace:
Sep 18 19:37:59 AMD-12 kernel: [380860.166676]  __schedule+0x291/0x8a0
Sep 18 19:37:59 AMD-12 kernel: [380860.166678]  schedule+0x2c/0x80
Sep 18 19:37:59 AMD-12 kernel: [380860.166681]  io_schedule+0x16/0x40
Sep 18 19:37:59 AMD-12 kernel: [380860.166684]  wait_on_page_bit_common+0xd8/0x160
Sep 18 19:37:59 AMD-12 kernel: [380860.166686]  ? page_cache_tree_insert+0xe0/0xe0
Sep 18 19:37:59 AMD-12 kernel: [380860.166688]  __filemap_fdatawait_range+0xfa/0x160
Sep 18 19:37:59 AMD-12 kernel: [380860.166691]  filemap_fdatawait_keep_errors+0x1e/0x40
Sep 18 19:37:59 AMD-12 kernel: [380860.166694]  jbd2_journal_commit_transaction+0x6e1/0x1720
Sep 18 19:37:59 AMD-12 kernel: [380860.166696]  ? __switch_to_asm+0x34/0x70
Sep 18 19:37:59 AMD-12 kernel: [380860.166698]  ? __switch_to_asm+0x34/0x70
Sep 18 19:37:59 AMD-12 kernel: [380860.166701]  kjournald2+0xc8/0x270
Sep 18 19:37:59 AMD-12 kernel: [380860.166702]  ? kjournald2+0xc8/0x270
Sep 18 19:37:59 AMD-12 kernel: [380860.166704]  ? wait_woken+0x80/0x80
Sep 18 19:37:59 AMD-12 kernel: [380860.166707]  kthread+0x121/0x140
Sep 18 19:37:59 AMD-12 kernel: [380860.166709]  ? commit_timeout+0x20/0x20
Sep 18 19:37:59 AMD-12 kernel: [380860.166710]  ? kthread_create_worker_on_cpu+0x70/0x70
Sep 18 19:37:59 AMD-12 kernel: [380860.166713]  ? do_syscall_64+0x73/0x130
Sep 18 19:37:59 AMD-12 kernel: [380860.166715]  ? SyS_exit_group+0x14/0x20
Sep 18 19:37:59 AMD-12 kernel: [380860.166716]  ret_from_fork+0x22/0x40
Sep 18 19:37:59 AMD-12 kernel: [380860.166784] INFO: task kworker/u256:2:53420 blocked for more than 120 seconds.
Sep 18 19:37:59 AMD-12 kernel: [380860.166788]       Tainted: P        W  OE    4.15.0-45-generic #48-Ubuntu
Sep 18 19:37:59 AMD-12 kernel: [380860.166790] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 18 19:37:59 AMD-12 kernel: [380860.166793] kworker/u256:2  D    0 53420      2 0x80000000
Sep 18 19:37:59 AMD-12 kernel: [380860.166798] Workqueue: writeback wb_workfn (flush-8:0)
Sep 18 19:37:59 AMD-12 kernel: [380860.166799] Call Trace:
Sep 18 19:37:59 AMD-12 kernel: [380860.166801]  __schedule+0x291/0x8a0
Sep 18 19:37:59 AMD-12 kernel: [380860.166803]  schedule+0x2c/0x80
Sep 18 19:37:59 AMD-12 kernel: [380860.166804]  io_schedule+0x16/0x40
Sep 18 19:37:59 AMD-12 kernel: [380860.166806]  get_request+0x2a4/0x800
Sep 18 19:37:59 AMD-12 kernel: [380860.166808]  ? wait_woken+0x80/0x80
Sep 18 19:37:59 AMD-12 kernel: [380860.166810]  blk_queue_bio+0x128/0x450
Sep 18 19:37:59 AMD-12 kernel: [380860.166811]  generic_make_request+0x124/0x300
Sep 18 19:37:59 AMD-12 kernel: [380860.166813]  submit_bio+0x73/0x140
Sep 18 19:37:59 AMD-12 kernel: [380860.166814]  ? submit_bio+0x73/0x140
Sep 18 19:37:59 AMD-12 kernel: [380860.166816]  ext4_io_submit+0x4c/0x60
Sep 18 19:37:59 AMD-12 kernel: [380860.166817]  ext4_bio_write_page+0x262/0x530
Sep 18 19:37:59 AMD-12 kernel: [380860.166820]  mpage_submit_page+0x59/0x80
Sep 18 19:37:59 AMD-12 kernel: [380860.166822]  mpage_map_and_submit_buffers+0x133/0x230
Sep 18 19:37:59 AMD-12 kernel: [380860.166824]  ext4_writepages+0x8ed/0xf40
Sep 18 19:37:59 AMD-12 kernel: [380860.166827]  do_writepages+0x4b/0xe0
Sep 18 19:37:59 AMD-12 kernel: [380860.166829]  ? do_writepages+0x4b/0xe0
Sep 18 19:37:59 AMD-12 kernel: [380860.166831]  ? fprop_fraction_percpu+0x2f/0x80
Sep 18 19:37:59 AMD-12 kernel: [380860.166833]  ? ktime_get+0x43/0xa0
Sep 18 19:37:59 AMD-12 kernel: [380860.166835]  __writeback_single_inode+0x45/0x340
Sep 18 19:37:59 AMD-12 kernel: [380860.166836]  ? __writeback_single_inode+0x45/0x340
Sep 18 19:37:59 AMD-12 kernel: [380860.166837]  writeback_sb_inodes+0x1e1/0x510
Sep 18 19:37:59 AMD-12 kernel: [380860.166839]  __writeback_inodes_wb+0x67/0xb0
Sep 18 19:37:59 AMD-12 kernel: [380860.166841]  wb_writeback+0x271/0x300
Sep 18 19:37:59 AMD-12 kernel: [380860.166843]  wb_workfn+0x1bb/0x400
Sep 18 19:37:59 AMD-12 kernel: [380860.166844]  ? wb_workfn+0x1bb/0x400
Sep 18 19:37:59 AMD-12 kernel: [380860.166846]  process_one_work+0x1de/0x410
Sep 18 19:37:59 AMD-12 kernel: [380860.166847]  worker_thread+0x32/0x410
Sep 18 19:37:59 AMD-12 kernel: [380860.166849]  kthread+0x121/0x140
Sep 18 19:37:59 AMD-12 kernel: [380860.166850]  ? process_one_work+0x410/0x410
Sep 18 19:37:59 AMD-12 kernel: [380860.166852]  ? kthread_create_worker_on_cpu+0x70/0x70
Sep 18 19:37:59 AMD-12 kernel: [380860.166854]  ? do_syscall_64+0x73/0x130
Sep 18 19:37:59 AMD-12 kernel: [380860.166855]  ? SyS_exit_group+0x14/0x20
Sep 18 19:37:59 AMD-12 kernel: [380860.166856]  ret_from_fork+0x22/0x40
Sep 18 19:39:11 AMD-12 kernel: [380931.924114] megaraid_sas 0000:49:00.0: Diag reset adapter never cleared megasas_adp_reset_fusion 3782
Sep 18 19:39:11 AMD-12 kernel: [380931.924116] megaraid_sas 0000:49:00.0: Reset failed, killing adapter scsi0.
Sep 18 19:39:12 AMD-12 kernel: [380932.932854] sd 0:2:0:0: Device offlined - not ready after error recovery
Sep 18 19:39:12 AMD-12 kernel: [380932.932873] sd 0:2:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.932875] sd 0:2:0:0: [sda] tag#0 CDB: Read(16) 88 00 00 00 00 09 d7 31 12 c8 00 00 01 c0 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.932877] print_req_error: I/O error, dev sda, sector 42265023176
Sep 18 19:39:12 AMD-12 kernel: [380932.932909] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.932913] sd 0:2:0:0: [sda] killing request
Sep 18 19:39:12 AMD-12 kernel: [380932.932921] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.932933] sd 0:2:0:0: [sda] tag#3 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.932935] sd 0:2:0:0: [sda] tag#3 CDB: Read(16) 88 00 00 00 00 09 d7 30 9c a8 00 00 02 00 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.932936] print_req_error: I/O error, dev sda, sector 42264992936
Sep 18 19:39:12 AMD-12 kernel: [380932.932956] sd 0:2:0:0: [sda] FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.932959] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.932969] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283119424)
Sep 18 19:39:12 AMD-12 kernel: [380932.932972] Buffer I/O error on device sda1, logical block 5283118970
Sep 18 19:39:12 AMD-12 kernel: [380932.932979] Buffer I/O error on device sda1, logical block 5283118971
Sep 18 19:39:12 AMD-12 kernel: [380932.932982] Buffer I/O error on device sda1, logical block 5283118972
Sep 18 19:39:12 AMD-12 kernel: [380932.932985] sd 0:2:0:0: [sda] CDB: Read(16) 88 00 00 00 00 09 d7 31 21 e0 00 00 01 00 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.932986] Buffer I/O error on device sda1, logical block 5283118973
Sep 18 19:39:12 AMD-12 kernel: [380932.932990] Buffer I/O error on device sda1, logical block 5283118974
Sep 18 19:39:12 AMD-12 kernel: [380932.932991] Buffer I/O error on device sda1, logical block 5283118975
Sep 18 19:39:12 AMD-12 kernel: [380932.932992] Buffer I/O error on device sda1, logical block 5283118976
Sep 18 19:39:12 AMD-12 kernel: [380932.932993] Buffer I/O error on device sda1, logical block 5283118977
Sep 18 19:39:12 AMD-12 kernel: [380932.932995] Buffer I/O error on device sda1, logical block 5283118978
Sep 18 19:39:12 AMD-12 kernel: [380932.932997] Buffer I/O error on device sda1, logical block 5283118979
Sep 18 19:39:12 AMD-12 kernel: [380932.933009] print_req_error: I/O error, dev sda, sector 42265027040
Sep 18 19:39:12 AMD-12 kernel: [380932.933075] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283119425)
Sep 18 19:39:12 AMD-12 kernel: [380932.933099] sd 0:2:0:0: [sda] tag#2 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.933101] sd 0:2:0:0: [sda] tag#2 CDB: Write(16) 8a 00 00 00 00 09 d7 2f dc 98 00 00 00 08 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.933102] print_req_error: I/O error, dev sda, sector 42264943768
Sep 18 19:39:12 AMD-12 kernel: [380932.933107] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283117972)
Sep 18 19:39:12 AMD-12 kernel: [380932.933108] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.933113] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283120514)
Sep 18 19:39:12 AMD-12 kernel: [380932.933131] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.933144] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283120552)
Sep 18 19:39:12 AMD-12 kernel: [380932.933179] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.933188] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283120557)
Sep 18 19:39:12 AMD-12 kernel: [380932.933204] sd 0:2:0:0: [sda] tag#4 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.933206] sd 0:2:0:0: [sda] tag#4 CDB: Write(16) 8a 00 00 00 00 09 d7 2f df 70 00 00 03 30 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.933207] print_req_error: I/O error, dev sda, sector 42264944496
Sep 18 19:39:12 AMD-12 kernel: [380932.933212] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283118163)
Sep 18 19:39:12 AMD-12 kernel: [380932.933247] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283118164)
Sep 18 19:39:12 AMD-12 kernel: [380932.933255] sd 0:2:0:0: [sda] tag#5 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.933258] sd 0:2:0:0: [sda] tag#5 CDB: Write(16) 8a 00 00 00 00 09 d7 2f e7 48 00 00 00 08 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.933264] print_req_error: I/O error, dev sda, sector 42264946504
Sep 18 19:39:12 AMD-12 kernel: [380932.933272] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283118314)
Sep 18 19:39:12 AMD-12 kernel: [380932.933277] sd 0:2:0:0: [sda] tag#7 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.933279] sd 0:2:0:0: [sda] tag#7 CDB: Write(16) 8a 00 00 00 00 09 d7 2f e8 a0 00 00 00 08 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.933280] print_req_error: I/O error, dev sda, sector 42264946848
Sep 18 19:39:12 AMD-12 kernel: [380932.933286] EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 107872263 (offset 0 size 0 starting block 5283118357)
Sep 18 19:39:12 AMD-12 kernel: [380932.933297] sd 0:2:0:0: [sda] tag#8 FAILED Result: hostbyte=DID_REQUEUE driverbyte=DRIVER_OK
Sep 18 19:39:12 AMD-12 kernel: [380932.933299] sd 0:2:0:0: [sda] tag#8 CDB: Write(16) 8a 00 00 00 00 09 d7 2f e9 70 00 00 02 88 00 00
Sep 18 19:39:12 AMD-12 kernel: [380932.933299] print_req_error: I/O error, dev sda, sector 42264947056
Sep 18 19:39:12 AMD-12 kernel: [380932.933333] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.958414] EXT4-fs error (device sda1): ext4_wait_block_bitmap:524: comm kworker/u256:2: Cannot read block bitmap - block_group = 173090, block_bitmap = 5671747586
Sep 18 19:39:12 AMD-12 kernel: [380932.958536] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.960881] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.960966] JBD2: Detected IO errors while flushing file data on sda1-8
Sep 18 19:39:12 AMD-12 kernel: [380932.960986] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.961003] EXT4-fs (sda1): Delayed block allocation failed for inode 149618765 at logical offset 434176 with max blocks 1024 with error 5
Sep 18 19:39:12 AMD-12 kernel: [380932.961004] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.961013] Aborting journal on device sda1-8.
Sep 18 19:39:12 AMD-12 kernel: [380932.961017] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.961022] JBD2: Error -5 detected when updating journal superblock for sda1-8.
Sep 18 19:39:12 AMD-12 kernel: [380932.961024] EXT4-fs (sda1): This should not happen!! Data will be lost
Sep 18 19:39:12 AMD-12 kernel: [380932.961024] 
Sep 18 19:39:12 AMD-12 kernel: [380932.961028] EXT4-fs error (device sda1) in ext4_writepages:2897: IO failure
Sep 18 19:39:12 AMD-12 kernel: [380932.961034] EXT4-fs (sda1): previous I/O error to superblock detected
Sep 18 19:39:12 AMD-12 kernel: [380932.961041] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:12 AMD-12 kernel: [380932.961061] JBD2: Detected IO errors while flushing file data on sda1-8
Sep 18 19:39:16 AMD-12 systemd[1]: Stopping User Manager for UID 0...
Sep 18 19:39:16 AMD-12 systemd[52307]: Stopped target Default.
Sep 18 19:39:16 AMD-12 systemd[52307]: Stopped target Basic System.
Sep 18 19:39:16 AMD-12 systemd[52307]: Stopped target Timers.
Sep 18 19:39:16 AMD-12 systemd[52307]: Stopped target Paths.
Sep 18 19:39:16 AMD-12 systemd[52307]: Stopped target Sockets.
Sep 18 19:39:16 AMD-12 systemd[52307]: Closed GnuPG cryptographic agent (ssh-agent emulation).
Sep 18 19:39:16 AMD-12 systemd[52307]: Closed GnuPG network certificate management daemon.
Sep 18 19:39:16 AMD-12 systemd[52307]: Closed GnuPG cryptographic agent and passphrase cache.
Sep 18 19:39:16 AMD-12 systemd[52307]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
Sep 18 19:39:16 AMD-12 systemd[52307]: Closed GnuPG cryptographic agent and passphrase cache (access for web browsers).
Sep 18 19:39:16 AMD-12 systemd[52307]: Reached target Shutdown.
Sep 18 19:39:16 AMD-12 systemd[52307]: Starting Exit the Session...
Sep 18 19:39:17 AMD-12 systemd[52307]: Received SIGRTMIN+24 from PID 53447 (kill).
Sep 18 19:39:17 AMD-12 systemd[1]: Stopped User Manager for UID 0.
Sep 18 19:39:17 AMD-12 systemd[1]: Removed slice User Slice of root.
Sep 18 19:39:42 AMD-12 kernel: [380963.587204] EXT4-fs (sda1): previous I/O error to superblock detected
Sep 18 19:39:42 AMD-12 kernel: [380963.587237] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:42 AMD-12 kernel: [380963.587254] EXT4-fs error (device sda1): ext4_journal_check_start:61: Detected aborted journal
Sep 18 19:39:42 AMD-12 kernel: [380963.587258] EXT4-fs (sda1): Remounting filesystem read-only
Sep 18 19:39:42 AMD-12 kernel: [380963.587264] EXT4-fs (sda1): previous I/O error to superblock detected
Sep 18 19:39:42 AMD-12 kernel: [380963.587271] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 19:39:42 AMD-12 kernel: [380963.587298] EXT4-fs (sda1): ext4_writepages: jbd2_start: 1024 pages, ino 149618765; err -30
Sep 18 19:45:01 AMD-12 CRON[53455]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 19:55:01 AMD-12 CRON[53460]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:05:01 AMD-12 CRON[53466]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:15:01 AMD-12 CRON[53471]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:17:01 AMD-12 CRON[53474]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep 18 20:21:21 AMD-12 systemd[1]: Created slice User Slice of root.
Sep 18 20:21:21 AMD-12 systemd[1]: Starting User Manager for UID 0...
Sep 18 20:21:21 AMD-12 systemd[1]: Started Session 787 of user root.
Sep 18 20:21:21 AMD-12 systemd[53479]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Sep 18 20:21:21 AMD-12 systemd[53479]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Sep 18 20:21:21 AMD-12 systemd[53479]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers).
Sep 18 20:21:21 AMD-12 systemd[53479]: Reached target Paths.
Sep 18 20:21:21 AMD-12 systemd[53479]: Listening on GnuPG network certificate management daemon.
Sep 18 20:21:21 AMD-12 systemd[53479]: Listening on GnuPG cryptographic agent and passphrase cache.
Sep 18 20:21:21 AMD-12 systemd[53479]: Reached target Sockets.
Sep 18 20:21:21 AMD-12 systemd[53479]: Reached target Timers.
Sep 18 20:21:21 AMD-12 systemd[53479]: Reached target Basic System.
Sep 18 20:21:21 AMD-12 systemd[53479]: Reached target Default.
Sep 18 20:21:21 AMD-12 systemd[53479]: Startup finished in 35ms.
Sep 18 20:21:21 AMD-12 systemd[1]: Started User Manager for UID 0.
Sep 18 20:21:42 AMD-12 systemd[1]: Stopping User Manager for UID 0...
Sep 18 20:21:42 AMD-12 systemd[53479]: Stopped target Default.
Sep 18 20:21:42 AMD-12 systemd[53479]: Stopped target Basic System.
Sep 18 20:21:42 AMD-12 systemd[53479]: Stopped target Paths.
Sep 18 20:21:42 AMD-12 systemd[53479]: Stopped target Sockets.
Sep 18 20:21:42 AMD-12 systemd[53479]: Closed GnuPG cryptographic agent and passphrase cache.
Sep 18 20:21:42 AMD-12 systemd[53479]: Closed GnuPG cryptographic agent (ssh-agent emulation).
Sep 18 20:21:42 AMD-12 systemd[53479]: Stopped target Timers.
Sep 18 20:21:42 AMD-12 systemd[53479]: Closed GnuPG network certificate management daemon.
Sep 18 20:21:42 AMD-12 systemd[53479]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
Sep 18 20:21:42 AMD-12 systemd[53479]: Closed GnuPG cryptographic agent and passphrase cache (access for web browsers).
Sep 18 20:21:42 AMD-12 systemd[53479]: Reached target Shutdown.
Sep 18 20:21:42 AMD-12 systemd[53479]: Starting Exit the Session...
Sep 18 20:21:42 AMD-12 systemd[53479]: Received SIGRTMIN+24 from PID 53517 (kill).
Sep 18 20:21:42 AMD-12 systemd[1]: Stopped User Manager for UID 0.
Sep 18 20:21:42 AMD-12 systemd[1]: Removed slice User Slice of root.
Sep 18 20:24:54 AMD-12 systemd[1]: Created slice User Slice of root.
Sep 18 20:24:54 AMD-12 systemd[1]: Starting User Manager for UID 0...
Sep 18 20:24:54 AMD-12 systemd[1]: Started Session 789 of user root.
Sep 18 20:24:54 AMD-12 systemd[53522]: Reached target Timers.
Sep 18 20:24:54 AMD-12 systemd[53522]: Listening on GnuPG network certificate management daemon.
Sep 18 20:24:54 AMD-12 systemd[53522]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Sep 18 20:24:54 AMD-12 systemd[53522]: Listening on GnuPG cryptographic agent and passphrase cache.
Sep 18 20:24:54 AMD-12 systemd[53522]: Reached target Paths.
Sep 18 20:24:54 AMD-12 systemd[53522]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Sep 18 20:24:54 AMD-12 systemd[53522]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers).
Sep 18 20:24:54 AMD-12 systemd[53522]: Reached target Sockets.
Sep 18 20:24:54 AMD-12 systemd[53522]: Reached target Basic System.
Sep 18 20:24:54 AMD-12 systemd[53522]: Reached target Default.
Sep 18 20:24:54 AMD-12 systemd[53522]: Startup finished in 19ms.
Sep 18 20:24:54 AMD-12 systemd[1]: Started User Manager for UID 0.
Sep 18 20:25:01 AMD-12 CRON[53548]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:25:20 AMD-12 kernel: [383701.724908] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 20:25:20 AMD-12 kernel: [383701.724940] EXT4-fs warning: 113 callbacks suppressed
Sep 18 20:25:20 AMD-12 kernel: [383701.724942] EXT4-fs warning (device sda1): htree_dirblock_to_tree:977: inode #2: lblock 0: comm ls: error -5 reading directory block
Sep 18 20:25:34 AMD-12 kernel: [383715.119956] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 20:25:34 AMD-12 kernel: [383715.119979] EXT4-fs error (device sda1): ext4_find_entry:1436: inode #2: comm bash: reading directory lblock 0
Sep 18 20:25:34 AMD-12 kernel: [383715.120003] EXT4-fs (sda1): previous I/O error to superblock detected
Sep 18 20:25:34 AMD-12 kernel: [383715.120011] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 20:25:34 AMD-12 kernel: [383715.121266] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 20:25:34 AMD-12 kernel: [383715.121278] EXT4-fs error (device sda1): ext4_find_entry:1436: inode #2: comm bash: reading directory lblock 0
Sep 18 20:25:34 AMD-12 kernel: [383715.121288] EXT4-fs (sda1): previous I/O error to superblock detected
Sep 18 20:25:34 AMD-12 kernel: [383715.121293] sd 0:2:0:0: rejecting I/O to offline device
Sep 18 20:35:01 AMD-12 CRON[53616]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:45:01 AMD-12 CRON[53620]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:55:01 AMD-12 CRON[53625]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Sep 18 20:59:22 AMD-12 systemd-resolved[1571]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.

Any kinds of help or solution will be widely welcomed!

Please help me if you have any ideas!

Thanks a lot!

filesystems
kernel
ubuntu-18.04
disk
raid
asked on Stack Overflow Sep 19, 2020 by user1119007

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0