These day,I found the one cpu load high
$sudo top
%Cpu0 : 7.0 us, 0.0 sy, 0.0 ni, 93.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu1 : 10.6 us, 0.0 sy, 0.0 ni, 0.0 id, 89.4 wa, 0.0 hi, 0.0 si, 0.0 st
...
and then use command isot to find out more info
devops@dn008:~$ sudo iostat -d -x 3
Device r/s w/s rkB/s wkB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
sda 81.33 96.67 1158.67 4352.00 0.00 0.00 0.00 0.00 6.30 1.06 0.61 14.25 45.02 1.30 23.20
sdb 83.67 102.67 1150.67 4334.67 0.00 0.00 0.00 0.00 4.13 0.90 0.44 13.75 42.22 1.07 19.87
sdc 82.00 108.00 1152.00 4330.67 0.00 0.00 0.00 0.00 4.63 0.96 0.49 14.05 40.10 1.11 21.07
...
dm-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.00 0.00 0.00 0.00 100.00
dm-5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-9 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.00 0.00 0.00 0.00 100.00
there have two block "dm-4" and "dm-9" that the util is high.
devops@dn008:~$ sudo ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 236 Oct 6 08:48 control
lrwxrwxrwx 1 root root 8 Nov 7 07:40 mysql_backup_lvm2_500g-tp_20201107 -> ../dm-12
lrwxrwxrwx 1 root root 8 Nov 7 07:40 mysql_backup_lvm2_500g-tp_20201107_tdata -> ../dm-11
lrwxrwxrwx 1 root root 8 Nov 7 07:40 mysql_backup_lvm2_500g-tp_20201107_tmeta -> ../dm-10
lrwxrwxrwx 1 root root 7 Oct 6 13:53 vg_a0fbb2c067364fc4d49dd33a5782c823-brick_2664622db7e4ac61df1ccf5a19d99032 -> ../dm-9
lrwxrwxrwx 1 root root 7 Oct 6 13:53 vg_a0fbb2c067364fc4d49dd33a5782c823-brick_467a0447a34476f2af26ea6edc45c153 -> ../dm-4
then I found this is a lvm block device,but the vg and lv info lost.
devops@dn008:~$ sudo gluster volume list |grep c823
devops@dn008:~$ sudo blktrace -d /dev/dm-9 -o - | blkparse -i -
devops@dn008:~$ sudo strace blktrace -d /dev/dm-9 -o dm-9.txt
futex(0x56453dd5f98c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1604906654, tv_nsec=896412000}, 0xffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x56453dd5f920, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x56453dd5f98c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1604906654, tv_nsec=946493000}, 0xffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x56453dd5f920, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x56453dd5f98c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1604906654, tv_nsec=996586000}, 0xffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x56453dd5f920, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x56453dd5f98c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1604906655, tv_nsec=46691000}, 0xffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x56453dd5f920, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x56453dd5f98c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1604906655, tv_nsec=96787000}, 0xffffffff^C) = ? ERESTART_RESTARTBLOCK (Interrupted by signal)
By the way, the command "fdisk -l " doesn't matter.
devops@dn008:~$ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
...
zd0 230:0 0 20G 0 disk
zd16 230:16 0 500G 0 disk
├─mysql_backup_lvm2_500g-tp_20201107_tmeta
│ 253:10 0 15.8G 0 lvm
│ └─mysql_backup_lvm2_500g-tp_20201107
│ 253:12 0 467.9G 0 lvm /glusterfs/glusterfs-thin-lvm-brick/mounts/mysql_backup_lvm2_500g/tp_20201107/brick
└─mysql_backup_lvm2_500g-tp_20201107_tdata
253:11 0 467.9G 0 lvm
└─mysql_backup_lvm2_500g-tp_20201107
253:12 0 467.9G 0 lvm /glusterfs/glusterfs-thin-lvm-brick/mounts/mysql_backup_lvm2_500g/tp_20201107/brick
so I guess that the mapping info lost between /dev/zd0 and /dev/mapper/vg_a0fbb2c067364fc4d49dd33a5782c823
devops@dn008:~$ sudo lvdisplay -a
--- Logical volume ---
LV Name tp_20201107
VG Name mysql_backup_lvm2_500g
LV UUID Gnb620-WiqH-W830-tZcE-xrnq-c8NO-dwoHPn
LV Write Access read/write
LV Creation host, time dn008.data.bm.pd.sz.deeproute.ai, 2020-11-07 07:40:04 +0000
LV Pool metadata tp_20201107_tmeta
LV Pool data tp_20201107_tdata
LV Status available
# open 1
LV Size 467.87 GiB
Allocated pool data 0.00%
Allocated metadata 0.11%
Current LE 119775
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:12
--- Logical volume ---
Internal LV Name lvol0_pmspare
VG Name mysql_backup_lvm2_500g
LV UUID p9BmRL-cuMi-P3gq-cG65-Wn3V-zLS1-4eBdlS
LV Write Access read/write
LV Creation host, time dn008.data.bm.pd.sz.deeproute.ai, 2020-11-07 07:40:04 +0000
LV Status NOT available
LV Size 15.81 GiB
Current LE 4048
Segments 1
Allocation inherit
Read ahead sectors auto
--- Logical volume ---
Internal LV Name tp_20201107_tmeta
VG Name mysql_backup_lvm2_500g
LV UUID jaE630-LPZm-of1H-1ypn-7Jro-ad8j-Ocde6I
LV Write Access read/write
LV Creation host, time dn008.data.bm.pd.sz.deeproute.ai, 2020-11-07 07:40:04 +0000
LV Status available
# open 1
LV Size 15.81 GiB
Current LE 4048
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:10
--- Logical volume ---
Internal LV Name tp_20201107_tdata
VG Name mysql_backup_lvm2_500g
LV UUID 6CpLK5-V3sJ-CseC-w9gr-gdhI-nEdp-ZibM7r
LV Write Access read/write
LV Creation host, time dn008.data.bm.pd.sz.deeproute.ai, 2020-11-07 07:40:04 +0000
LV Status available
# open 1
LV Size 467.87 GiB
Current LE 119775
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:11
the lvdisplay and vgdiplay could not find the info.
devops@dn008:~$ sudo dmsetup ls --tree
mysql_backup_lvm2_500g-tp_20201107 (253:12)
├─mysql_backup_lvm2_500g-tp_20201107_tdata (253:11)
│ └─ (230:16)
└─mysql_backup_lvm2_500g-tp_20201107_tmeta (253:10)
└─ (230:16)
vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153 (253:3)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153-tpool (253:2)
├─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153_tdata (253:1)
│ └─ (8:240)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153_tmeta (253:0)
└─ (8:240)
vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032 (253:8)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032-tpool (253:7)
├─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032_tdata (253:6)
│ └─ (8:240)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032_tmeta (253:5)
└─ (8:240)
vg_a0fbb2c067364fc4d49dd33a5782c823-brick_467a0447a34476f2af26ea6edc45c153 (253:4)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153-tpool (253:2)
├─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153_tdata (253:1)
│ └─ (8:240)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_467a0447a34476f2af26ea6edc45c153_tmeta (253:0)
└─ (8:240)
vg_a0fbb2c067364fc4d49dd33a5782c823-brick_2664622db7e4ac61df1ccf5a19d99032 (253:9)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032-tpool (253:7)
├─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032_tdata (253:6)
│ └─ (8:240)
└─vg_a0fbb2c067364fc4d49dd33a5782c823-tp_2664622db7e4ac61df1ccf5a19d99032_tmeta (253:5)
└─ (8:240)
so, I want to know how to remove the /dev/mapper devices without the info. the os verison is ubuntu18. thanks.
User contributions licensed under CC BY-SA 3.0