I was having the common issue of not being able to restart httpd because of a high number of locked semaphores. After clearing those and restarting httpd, I also doubled the semaphores.
However, I'm lead to believe some Apache process has a memory leak and it's not releasing them, so I'll still run into the issue but later.
Example ipcs -s
:
------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 13467648 apache 600 1
0x00000000 13697025 apache 600 1
0x00000000 13729794 apache 600 1
0x00000000 13762563 apache 600 1
0x00000000 13795332 apache 600 1
0x00000000 14057477 apache 600 1
0x00000000 14123014 apache 600 1
0x00000000 14155783 apache 600 1
0x00000000 14188552 apache 600 1
0x00000000 14221321 apache 600 1
0x00000000 14254090 apache 600 1
So let's trace down what process was managing those ipcs -s -i 13697025
:
Semaphore Array semid=13697025
uid=48 gid=48 cuid=0 cgid=0
mode=0600, access_perms=0600
nsems = 1
otime = Not set
ctime = Thu Jul 11 03:41:01 2019
semnum value ncount zcount pid
0 1 0 0 18395
And finally what corresponds to the pid ps --pid 18395
:
PID TTY TIME CMD
So am I reading that right the second semaphore in the list belongs to a process that already died and didn't clean up?
For example, running through the process with the second to last in the list produced that it actually did belong to a running process:
PID TTY TIME CMD
22331 ? 00:03:42 httpd
It's obvious these are owned by Apache, but what would be the best way of debugging what's causing this. We have in the range of 50+ virtual hosts is there a way to log an actual server request and trace it back to the process spawned, etc?
I have reason to believe it's the php-sqlsrv.x86_64
extension or something related to it because of the time relation between when that was configured and when the issue started happening. I have the ability to roll back but just want to learn how to go deep into debugging, perhaps even to submit a patch for the bug.
User contributions licensed under CC BY-SA 3.0