I have a Linux server with 128 GB of RAM and several KVM virtual machines and the usage of the memory usage of a server is much more than I expected. All virtual machines are using about 32 gigabytes of memory in total, but when I checking memory usage with the free -m
command, it reports memory usage over 80 gigabytes. How to find out which programs use extra memory?
There is no significant shared memory usage.
# ipcs -m --human
------ Shared Memory Segments --------
key shmid owner perms size nattch status
0x00000000 0 root 644 80B 2
0x00000000 32769 root 644 16K 2
0x00000000 65538 root 644 280B 2
0x00000000 163843 zabbix 600 576B 6 dest
0x00000000 196612 zabbix 600 1.2M 6 dest
I've tried to sum the memory usage by ps
command, it gives me roughly 40 gigabytes.
# ps -o rss ax | perl -nle '$sum += $_ } END { print $sum'
40706900
But free -m
reports that memory usage is over 95 gigabytes (used-buff/cache).
# free -m
total used free shared buff/cache available
Mem: 128900 112804 1853 1690 14242 13355
Swap: 0 0 0
On the other hand tmpfs
says that there is 63 GB of free memory.
# df -h
Filesystem Size Used Avail Use% Mounted on
...
tmpfs 63G 57M 63G 1% /dev/shm
...
Is there some way to find out how much free memory do I have?
UPD: Found a bunch of answers, unreclaimable slabs (50GB), and ZFS ARC cache (6GB). Both memory drains are well undocumented.
Slabs: https://serverfault.com/questions/877257/why-cant-my-linux-kernel-reclaim-its-slab-memory
ARC cache: https://askubuntu.com/questions/836246/general-way-to-monitor-all-memory-usages
User contributions licensed under CC BY-SA 3.0