Kernel panic issue: How to fix hung_task_timeout_secs and blocked for more than 300 seconds?

My server is CentOS7 64bit going under kernel panic issue and rebooted after everyday around 7 to 8am. Here is part of dmesg

[  536.606448] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[82341.807403] INFO: task sync:6493 blocked for more than 300 seconds.
[82341.807406]       Tainted: P           O 3.14.39ltsi-ayan-130.0.33 #1
[82341.807407] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[82341.807408] sync            D ffffffff81a0b3c0     0  6493      1 0x00000000
[82341.807410]  ffff8801ed563d80 0000000000000086 0000000000009000 ffff880308cb4800
[82341.807412]  ffff8803f962b000 ffff8801ed563fd8 ffff880308cb4800 ffff8803f922f600
[82341.807413]  ffff8801ed563d60 ffff8801ed563e98 7fffffffffffffff ffff8801ed563e90
[82341.807415] Call Trace:
[82341.807421]  [<ffffffff81802029>] schedule+0x29/0x70
[82341.807422]  [<ffffffff81801169>] schedule_timeout+0x199/0x280
[82341.807426]  [<ffffffff81085d42>] ? __queue_delayed_work+0xa2/0x1b0
.
.
.
[82341.807867] Code: 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 65 48 8b 04 25 70 a8 00 00 48 8b 80 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e0 0f 01 c9 <65> 48 8b 04 25 70 a8 00 00 83 a0 3c c0 ff ff fb 0f ae f0 65 48 
[82341.808450] Kernel panic - not syncing: hung_task: blocked tasks
[82341.814824] CPU: 0 PID: 75 Comm: khungtaskd Tainted: P           O 3.14.39ltsi-ayan-130.0.33 #1

What I understood from dmesg, sync is a process with pid-6490 waiting for DISK operation and blocked for 300s because DISK is hogging by other process. sync goes under process state D (uninterruptible sleep).

Year back when first time I saw panic issue resolved by tweaking sysctl.conf file then issue went away. Guided by website https://www.blackmoreops.com/2014/09/22/linux-kernel-panic-issue-fix-hung_task_timeout_secs-blocked-120-seconds-problem/

Here is current config of /etc/sysctl.conf

kernel.hung_task_timeout_secs = 300
vm.dirty_background_ratio = 5 
vm.dirty_ratio = 10

I have configured daily cron job task for data backup of pgsql DB. Data size around 200GB and some other task.

Server details:

Filesystem                Size  Used Avail Use% Mounted on
total                     3.7T  1.1T  2.6T  30%
RAM: 16GB