Java 9.0.1 OpenJdk crashed the JVM with problem with AdaptiveWeightedAverage

1

My application failed for a customer crashing the JVM

I have these details:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xb6b48b12, pid=31447, tid=31508
#
# JRE version: OpenJDK Runtime Environment (9.0+11) (build 9.0.1+11)
# Java VM: OpenJDK Client VM (9.0.1+11, mixed mode, serial gc, linux-)
# Problematic frame:
# V  [libjvm.so+0x1d9b12]  AdaptiveWeightedAverage::sample(float)+0x55
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /mnt/system/config/Apps/SongKong/songkong/hs_err_pid31447.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

Does it look familar to anyone, I have search both https://bugs.java.com/bugdatabase/ and https://bugs.openjdk.java.net/secure/Dashboard.jspa without finding anything.

Here is the start of the hs_err_id log file

    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  SIGSEGV (0xb) at pc=0xb6aacb12, pid=13032, tid=13074
    #
    # JRE version: OpenJDK Runtime Environment (9.0+11) (build 9.0.1+11)
    # Java VM: OpenJDK Client VM (9.0.1+11, mixed mode, serial gc, linux-)
    # Problematic frame:
    # V  [libjvm.so+0x1d9b12]  AdaptiveWeightedAverage::sample(float)+0x55
    #
    # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    #

    ---------------  S U M M A R Y ------------

    Command Line: -Duser.home=/mnt/disk1/share -Xms150m -Xmx900m -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog -Dorg.jboss.logging.provider=jdk -Djava.util.logging.config.class=com.jthink.songkong.logging.StandardLogging --add-opens=java.base/java.lang=ALL-UNNAMED lib/songkong-6.8.jar -r

    Host: Marvell PJ4Bv7 Processor rev 1 (v7l), 1 cores, 1G, Buildroot 2014.11-rc1
    Time: Fri Feb  7 01:50:35 2020 GMT elapsed time: 53621 seconds (0d 14h 53m 41s)

    ---------------  T H R E A D  ---------------

    Current thread (0xb6657d00):  VMThread "VM Thread" [stack: 0x7b7e9000,0x7b869000] [id=13074]

    Stack: [0x7b7e9000,0x7b869000],  sp=0x7b8677a0,  free space=505k
    Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
    V  [libjvm.so+0x1d9b12]  AdaptiveWeightedAverage::sample(float)+0x55

    [error occurred during error reporting (printing native stack), id 0xb]


    siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000a

    Register to memory mapping:

      r0  = 0x00000000
    0x00000000 is an unknown value

      r1  = 0x00000100
    0x00000100 is an unknown value

      r2  = 0x000001fc
    0x000001fc is an unknown value

      r3  = 0x00000023
    0x00000023 is an unknown value

      r4  = 0x783bf970
    0x783bf970 is an unknown value

      r5  = 0x00000000
    0x00000000 is an unknown value

      r6  = 0x0e6a0000
    0x0e6a0000 is an unknown value

      r7  = 0x00000000
    0x00000000 is an unknown value

      r8  = 0x783bf940
    0x783bf940 is an unknown value

      r9  = 0x0e6a0000
    0x0e6a0000 is an unknown value

      r10 = 0x7b8679ec
    0x7b8679ec is an unknown value

      fp  = 0xb6657c38
    0xb6657c38 is an unknown value

      r12 = 0xfffffefe
    0xfffffefe is an unknown value

      sp  = 0x7b8677a0
    0x7b8677a0 is an unknown value

      lr  = 0xb6c62daf
    0xb6c62daf: <offset 0x0038fdaf> in /mnt/system/config/Apps/SongKong/songkong/jre/lib/client/libjvm.so at 0xb68d3000

      pc  = 0xb6aacb12
    0xb6aacb12: <offset 0x001d9b12> in /mnt/system/config/Apps/SongKong/songkong/jre/lib/client/libjvm.so at 0xb68d3000



    Registers:
      r0  = 0x00000000
      r1  = 0x00000100
      r2  = 0x000001fc
      r3  = 0x00000023
      r4  = 0x783bf970
      r5  = 0x00000000
      r6  = 0x0e6a0000
      r7  = 0x00000000
      r8  = 0x783bf940
      r9  = 0x0e6a0000
      r10 = 0x7b8679ec
      fp  = 0xb6657c38
      r12 = 0xfffffefe
      sp  = 0x7b8677a0
      lr  = 0xb6c62daf
      pc  = 0xb6aacb12
      cpsr = 0x80000030

    Top of Stack: (sp=0x7b8677a0)
    0x7b8677a0:   00000000 404e0000 00000001 000b6448
    0x7b8677b0:   7b8677b8 b6c62daf b6d4c8b8 783bf910
    0x7b8677c0:   00000000 b660d800 7b867808 b6d1f000
    0x7b8677d0:   b662f310 b6c62f43 7b8679f6 b6cac9cc
    0x7b8677e0:   7b8677f0 b6d1f000 7b8679ec b6b1d987
    0x7b8677f0:   7b867820 7b8679f0 7b867800 b6a068a3
    0x7b867800:   00000400 7b867820 00000000 00000012
    0x7b867810:   b660d800 00000000 00000012 b660d800 

    Instructions: (pc=0xb6aacb12)
    0xb6aacaf2:   ee064603 eeb83a10 ee377a46 ee286ac7
    0xb6aacb02:   ee667a07 ee876a86 eec67a27 ee777aa7
    0xb6aacb12:   edc47a87 46bd7a00 8b02ecbd 2064bd98
    0xb6aacb22:   f9e1f1fd 6a00edd4 0000e7db 480642c8 

    VM_Operation (0x77c5daa0): GenCollectForAllocation, mode: safepoint, requested by thread 0x77a0d790


    ---------------  P R O C E S S  ---------------

    Java Threads: ( => current thread )
      0x742a7448 JavaThread "Thread-513" daemon [_thread_blocked, id=11706, stack(0x76a70000,0x76ac0000)]
      0x73db7458 JavaThread "Thread-510" daemon [_thread_blocked, id=11027, stack(0x773b6000,0x77406000)]
      0x732b66f0 JavaThread "Thread-509" daemon [_thread_blocked, id=10757, stack(0x77b70000,0x77bc0000)]
      0x762428e0 JavaThread "Thread-508" daemon [_thread_blocked, id=10130, stack(0x79310000,0x79360000)]
      0x74284f48 JavaThread "Thread-503" daemon [_thread_blocked, id=8094, stack(0x78210000,0x78260000)]
      0x76d60e10 JavaThread "qtp9318356-11036" [_thread_blocked, id=4101, stack(0x78410000,0x78460000)]
      0x762dc8f0 JavaThread "qtp9318356-11035" [_thread_blocked, id=4100, stack(0x79bc0000,0x79c10000)]
      0x783bf910 JavaThread "qtp9318356-9925" [_thread_blocked, id=22685, stack(0x78260000,0x782b0000)]
      0x732170a8 JavaThread "qtp9318356-5567" [_thread_blocked, id=9426, stack(0x76810000,0x76860000)]
      0x760e2708 JavaThread "qtp9318356-5564" [_thread_blocked, id=9423, stack(0x768b0000,0x76900000)]
      0x77a0d790 JavaThread "Worker:12" [_thread_blocked, id=18002, stack(0x77c10000,0x77c60000)]
      0x77a0daf0 JavaThread "pool-1639-thread-1" [_thread_blocked, id=17984, stack(0x77186000,0x771d6000)]
      0x76db5e88 JavaThread "Worker:11" [_thread_blocked, id=17983, stack(0x77316000,0x77366000)]
      0x783c9df8 JavaThread "Worker:10" [_thread_blocked, id=17982, stack(0x77406000,0x77456000)]
      0x77a25388 JavaThread "SongLoaderWorker:9" [_thread_blocked, id=17942, stack(0x77366000,0x773b6000)]
      0x76d6bce8 JavaThread "com.jthink.songkong.server.callback.ServerFixSongs:8" [_thread_blocked, id=17922, stack(0x793b0000,0x79400000)]
      0x769c7cb8 JavaThread "pool-1636-thread-2" [_thread_blocked, id=17764, stack(0x77276000,0x772c6000)]
      0x732c50d8 JavaThread "pool-1636-thread-1" [_thread_blocked, id=17763, stack(0x77036000,0x77086000)]
      0x7690e140 JavaThread "pool-1601-thread-2" [_thread_blocked, id=11810, stack(0x771d6000,0x77226000)]
      0x769077c8 JavaThread "pool-1601-thread-1" [_thread_blocked, id=11809, stack(0x77226000,0x77276000)]
      0x769368a8 JavaThread "MappedByteBufferSource Async Flush Thread" daemon [_thread_blocked, id=8920, stack(0x76fe6000,0x77036000)]
      0x77d9d990 JavaThread "com.jthink.songkong.fileloader.LoadFolderWorker:3" [_thread_blocked, id=380, stack(0x77eb0000,0x77f00000)]
      0x77d940d8 JavaThread "pool-429-thread-1" [_thread_blocked, id=374, stack(0x77e60000,0x77eb0000)]
      0x77d9c740 JavaThread "SongLoaderCountFiles:2" [_thread_blocked, id=373, stack(0x77cb0000,0x77d00000)]
      0x7b670218 JavaThread "qtp9318356-91" [_thread_blocked, id=32431, stack(0x79360000,0x793b0000)]
      0x7833f418 JavaThread "qtp9318356-90" [_thread_in_native, id=32430, stack(0x79db0000,0x79e00000)]
      0x7b6e7478 JavaThread "MVStore background writer async:/mnt/disk1/share/SongKong/Prefs/Database/Database.mv.db" daemon [_thread_blocked, id=27227, stack(0x79270000,0x792c0000)]
      0x7b670cf8 JavaThread "H2 File Lock Watchdog (Socket) async:/mnt/disk1/share/SongKong/Prefs/Database/Database.lock.db" daemon [_thread_blocked, id=27222, stack(0x7a61c000,0x7a66c000)]
      0x7b694750 JavaThread "Scheduler-7136558" [_thread_blocked, id=13763, stack(0x78560000,0x785b0000)]
      0x7b6786c0 JavaThread "Scheduler-14033461" [_thread_blocked, id=13428, stack(0x785b0000,0x78600000)]
      0x7b6717d8 JavaThread "qtp9318356-45-acceptor-0@4cfe9c-ServerConnector@8b18a5{HTTP/1.1,[http/1.1]}{0.0.0.0:4567}" [_thread_in_native, id=13427, stack(0x7904c000,0x7909c000)]
      0x789c3e20 JavaThread "Thread-10" [_thread_blocked, id=13405, stack(0x791d0000,0x79220000)]
      0x7a0b4f10 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-HelperThread-#2" daemon [_thread_blocked, id=13275, stack(0x7a66c000,0x7a6bc000)]
      0x7a0b4958 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-HelperThread-#1" daemon [_thread_blocked, id=13274, stack(0x79b70000,0x79bc0000)]
      0x7a0b43c0 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-HelperThread-#0" daemon [_thread_blocked, id=13273, stack(0x79920000,0x79970000)]
      0x7a0feb80 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-AdminTaskTimer" daemon [_thread_blocked, id=13272, stack(0x79c10000,0x79c60000)]
      0x7adb1358 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=13146, stack(0x7a7e3000,0x7a833000)]
      0xb66f12d8 JavaThread "Service Thread" daemon [_thread_blocked, id=13114, stack(0x7b2b6000,0x7b306000)]
      0xb66a1a20 JavaThread "Common-Cleaner" daemon [_thread_blocked, id=13105, stack(0x7b306000,0x7b356000)]
      0xb6675c88 JavaThread "Sweeper thread" daemon [_thread_blocked, id=13104, stack(0x7b356000,0x7b3a6000)]
      0xb6674978 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=13103, stack(0x7b3a6000,0x7b426000)]
      0xb666cdc8 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=13102, stack(0x7b426000,0x7b476000)]
      0xb6667688 JavaThread "Finalizer" daemon [_thread_blocked, id=13099, stack(0x7b748000,0x7b798000)]
      0xb665a900 JavaThread "Reference Handler" daemon [_thread_blocked, id=13088, stack(0x7b798000,0x7b7e8000)]
      0xb6609308 JavaThread "main" [_thread_blocked, id=13037, stack(0xb67d9000,0xb6828000)]

    Other Threads:
    =>0xb6657d00 VMThread "VM Thread" [stack: 0x7b7e9000,0x7b869000] [id=13074]
      0xb66f2a90 WatcherThread [stack: 0x7b236000,0x7b2b6000] [id=13115]

    Threads with active compile tasks:

VM state:at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0xb6608520] Threads_lock - owner thread: 0xb6657d00
[0xb6608930] Heap_lock - owner thread: 0x77a0d790

Heap:
 def new generation   total 265600K, used 265599K [0x7c000000, 0x8e020000, 0x8ec00000)
  eden space 236160K, 100% used [0x7c000000, 0x8a6a0000, 0x8a6a0000)
  from space 29440K,  99% used [0x8c360000, 0x8e01fff8, 0x8e020000)
  to   space 29440K,   0% used [0x8a6a0000, 0x8a6a0000, 0x8c360000)
 tenured generation   total 589672K, used 298477K [0x8ec00000, 0xb2bda000, 0xb4400000)
   the space 589672K,  50% used [0x8ec00000, 0xa0f7b5c0, 0xa0f7b600, 0xb2bda000)
 Metaspace       used 45545K, capacity 46775K, committed 46976K, reserved 47408K

Card table byte_map: [0x7be3d000,0x7c000000] byte_map_base: 0x7ba5d000

Polling page: 0xb6f30000
java
crash
asked on Stack Overflow Feb 6, 2020 by Paul Taylor • edited Feb 7, 2020 by Paul Taylor

2 Answers

1

But surely the question is what does AdaptiveWeightedAverage do ?

If this was the intended question. Then you might have a look into the source.

hotspot/src/share/vm/gc/shared/gcUtil.cpp

// A weighted average maintains a running, weighted average
// of some float value (templates would be handy here if we
// need different types).
//
// The average is adaptive in that we smooth it for the
// initial samples; we don't use the weight until we have
// enough samples for it to be meaningful.
//
// This serves as our best estimate of a future unknown.
//
class AdaptiveWeightedAverage : public CHeapObj<mtGC> {
 private:
  float            _average;        // The last computed average
  unsigned         _sample_count;   // How often we've sampled this average
  unsigned         _weight;         // The weight used to smooth the averages
                                    //   A higher weight favors the most
                                    //   recent data.
  bool             _is_old;         // Has enough historical data

hotspot/src/share/vm/gc/shared/gcUtil.hpp

void AdaptiveWeightedAverage::sample(float new_sample) {
  increment_count();

  // Compute the new weighted average
  float new_avg = compute_adaptive_average(new_sample, average());
  set_average(new_avg);
  _last_sample = new_sample;
}

and

float AdaptiveWeightedAverage::compute_adaptive_average(float new_sample,
                                                        float average) {
  // We smooth the samples by not using weight() directly until we've
  // had enough data to make it meaningful. We'd like the first weight
  // used to be 1, the second to be 1/2, etc until we have
  // OLD_THRESHOLD/weight samples.

It's used by the garbage collector.

Maybe somehow related to a problem with available memory. (it's a guess, don't have much experience with Java on ARM platform)

This lead me to this guess max heap -Xmx900m vs system memory 1G.

answered on Stack Overflow Feb 18, 2020 by SubOptimal
0

The way out of this was to switch to Java 11 build, this no longer crashes with this error.

answered on Stack Overflow Apr 11, 2020 by Paul Taylor

User contributions licensed under CC BY-SA 3.0