I have an Akka program with threads that are spending most of their time in the futex_
state.
This is from an Akka program that isn't doing much. I am not familiar with the internals of Akka, so I'm not sure if this is normal or not.
F S UID PID PPID LWP C PRI NI ADDR SZ WCHAN TTY TIME CMD
4 S 1 19 0 19 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
5 S 1 19 0 25 0 80 0 - 1149556 futex_ ? 0:04 /usr/bin/java
1 S 1 19 0 26 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 27 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 28 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 29 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 30 1 80 0 - 1149556 futex_ ? 0:12 /usr/bin/java
1 S 1 19 0 31 0 80 0 - 1149556 futex_ ? 0:04 /usr/bin/java
1 S 1 19 0 32 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 33 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 43 0 80 0 - 1149556 futex_ ? 0:02 /usr/bin/java
1 S 1 19 0 44 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 45 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 46 0 80 0 - 1149556 futex_ ? 0:01 /usr/bin/java
1 S 1 19 0 47 0 80 0 - 1149556 futex_ ? 0:02 /usr/bin/java
1 S 1 19 0 50 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 53 0 80 0 - 1149556 futex_ ? 0:01 /usr/bin/java
1 S 1 19 0 60 0 80 0 - 1149556 futex_ ? 0:03 /usr/bin/java
1 S 1 19 0 121 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 122 0 80 0 - 1149556 futex_ ? 0:01 /usr/bin/java
1 S 1 19 0 123 0 80 0 - 1149556 ep_pol ? 0:00 /usr/bin/java
1 S 1 19 0 282 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 283 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 284 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 285 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 295 0 80 0 - 1149556 futex_ ? 0:01 /usr/bin/java
1 S 1 19 0 383 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
1 S 1 19 0 433 0 80 0 - 1149556 futex_ ? 0:01 /usr/bin/java
1 S 1 19 0 1109 0 80 0 - 1149556 futex_ ? 0:00 /usr/bin/java
Checking the syscalls, there are a ton of timed out futex calls constantly happening. Here's a snippet from just a fraction of a second:
[pid 43] futex(0x7f36266dfc28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 43] futex(0x7f36266dfc54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=950443844}, 0xffffffff <unfinished ...>
[pid 47] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 47] futex(0x7f3626f8ab28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 47] futex(0x7f3626f8ab54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=954449564}, 0xffffffff <unfinished ...>
[pid 33] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 33] futex(0x7f3624a11128, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 33] futex(0x7f3624a11154, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=998037087}, 0xffffffff <unfinished ...>
[pid 43] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 43] futex(0x7f36266dfc28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 43] futex(0x7f36266dfc54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=960733704}, 0xffffffff <unfinished ...>
[pid 47] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 47] futex(0x7f3626f8ab28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 47] futex(0x7f3626f8ab54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=964665830}, 0xffffffff <unfinished ...>
[pid 43] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 43] futex(0x7f36266dfc28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 43] futex(0x7f36266dfc54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=971001771}, 0xffffffff <unfinished ...>
[pid 47] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 47] futex(0x7f3626f8ab28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 47] futex(0x7f3626f8ab54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=973981797}, 0xffffffff <unfinished ...>
[pid 43] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 43] futex(0x7f36266dfc28, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 43] futex(0x7f36266dfc54, FUTEX_WAIT_BITSET_PRIVATE, 1, {tv_sec=14532, tv_nsec=980553240}, 0xffffffff <unfinished ...>
[pid 47] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
My theory is that this may be the Akka actors rapidly checking to see if there's any new work for them to do, but I am seeking confirmation if that is a valid theory. Thank you!
User contributions licensed under CC BY-SA 3.0