Heap Corruption In C When Using DLL With JNA

I am using C language Native API callbacks with DLL files. When we are calling callback first time everything is working fine but on second call I am getting heap corruption error and JVM is getting crashed.

In the native code the memory allocated in first call is being released and then is being used in second call again and during memory allocation in second call JVM is being crashed. But on the same place when in second call new memory pointer is used rather than the one which was used in previous call I am not getting this heap corruption error.

As this callback is called many times I can not keep on allocating new space every time. In below logs I am getting error as INVALID_POINTER_READ.

I am not able to understand what is the reason behind it and how this can be fixed. When same DLL is used with JNA it's working fine.

Java/JNA Code:

Setting Hook:

final PropertyCallBack callback = new PropertyCallBack();
final int setHookStatus = callback.setHook();


private static CALLBACK callback;

public int setHook() {

    if (callback != null) {
      return 0;
    }

    synchronized (this) {
      if (callback == null) {
        callback = new CALLBACK();
        return callback.setHook();
      }
    }

    return 0;
  }

Callback Method Called From Native:

@Override
  public int PropertyHook(final DESTINATION dest, final BACSTAC_READ_INFO.ByReference info) {

    final PROPERTY_CONTENTS.ByReference content = new PROPERTY_CONTENTS.ByReference();
    final BUFFER.ByReference buffer = new BUFFER.ByReference();

    // Memory assign
    final int bufferSize = 1048;
    buffer.pBuffer = new Memory(bufferSize);
    buffer.nBufferSize = bufferSize;
    content.tag = "INVALID";
    content.buffer = buffer;
    content.nElements = 0;

    Pointer dev = NativeLibrary.INSTANCE.Call_1();
    Pointer obj = null;
    if (dev != null) {
      obj = NativeLibrary.INSTANCE.call_2(dev, info.objectID);
    }

    final int readDbStatus = NativeLibrary.INSTANCE.call_3(obj, info.prop, info.index, content, null);

    final int responseStatus = NativeLibrary.INSTANCE.call_4(dest, info, content);

    return 0;
  }

When I analyzed heap dump with windbg I am getting below details:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(6201c.5ef10): Access violation - code c0000005 (first/second chance not available)
For analysis of this file, run !analyze -v
ntdll!NtWaitForMultipleObjects+0x14:
00007ffa`46deb4f4 c3              ret
0:026> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

*** WARNING: Unable to verify checksum for srv.dll
DEBUG_FLR_EXCEPTION_CODE(c0000374) and the ".exr -1" ExceptionCode(c0000005) don't match

KEY_VALUES_STRING: 1

    Key  : AV.Fault
    Value: Read

    Key  : Timeline.Process.Start.DeltaSec
    Value: 46

PROCESSES_ANALYSIS: 1
SERVICE_ANALYSIS: 1
STACKHASH_ANALYSIS: 1
TIMELINE_ANALYSIS: 1
Timeline: !analyze.Start
    Name: <blank>
    Time: 2019-12-02T11:13:41.439Z
    Diff: 3429439 mSec
Timeline: Dump.Current
    Name: <blank>
    Time: 2019-12-02T10:16:32.0Z
    Diff: 0 mSec
Timeline: Process.Start
    Name: <blank>
    Time: 2019-12-02T10:15:46.0Z
    Diff: 46000 mSec
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
CONTEXT:  (.ecxr)
rax=0000000000030000 rbx=000000002b200000 rcx=0000000000000303
rdx=0000000000000003 rsi=01fda8c00000ed00 rdi=000000002b223ef0
rip=00007ffa46d6cb7a rsp=000000002b8ff500 rbp=0000000000000008
 r8=0000000000000028  r9=0000000000000030 r10=00000000014da2d0
r11=00000000014e2ef0 r12=0000000000000001 r13=0000000000000003
r14=000000002b223ee0 r15=000000000600c1ba
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
ntdll!RtlpAllocateHeap+0xdaa:
00007ffa`46d6cb7a 498b07          mov     rax,qword ptr [r15] ds:00000000`0600c1ba=????????????????
Resetting default scope

FAULTING_IP: 
ntdll!RtlpAllocateHeap+daa
00007ffa`46d6cb7a 498b07          mov     rax,qword ptr [r15]

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007ffa46d6cb7a (ntdll!RtlpAllocateHeap+0x0000000000000daa)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: 000000000600c1ba
Attempt to read from address 000000000600c1ba

DEFAULT_BUCKET_ID:  HEAP_CORRUPTION

PROCESS_NAME:  javaw.exe

FOLLOWUP_IP: 
ntdll!RtlpAllocateHeap+daa
00007ffa`46d6cb7a 498b07          mov     rax,qword ptr [r15]

READ_ADDRESS:  000000000600c1ba 
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.
EXCEPTION_CODE: (NTSTATUS) 0xc0000374 - A heap has been corrupted.
EXCEPTION_CODE_STR:  c0000005
EXCEPTION_PARAMETER1:  0000000000000000
EXCEPTION_PARAMETER2:  000000000600c1ba
WATSON_BKT_PROCSTAMP:  5d1dea24
WATSON_BKT_PROCVER:  8.0.2210.11
PROCESS_VER_PRODUCT:  Java(TM) Platform SE 8
WATSON_BKT_MODULE:  ntdll.dll
WATSON_BKT_MODSTAMP:  7f828745
WATSON_BKT_MODOFFSET:  1cb7a
WATSON_BKT_MODVER:  10.0.17134.799
MODULE_VER_PRODUCT:  Microsoft® Windows® Operating System
BUILD_VERSION_STRING:  17134.1.amd64fre.rs4_release.180410-1804
MODLIST_WITH_TSCHKSUM_HASH:  f06ad8a6a7f7267c783c08e3a62df4696020d52f
MODLIST_SHA1_HASH:  cdafa8057ac19b1a3608c439ebbfa992407212d6
NTGLOBALFLAG:  0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS:  0
DUMP_FLAGS:  94
DUMP_TYPE:  1
ANALYSIS_SESSION_HOST:  MD2E86EC
ANALYSIS_SESSION_TIME:  12-02-2019 16:43:41.0439
ANALYSIS_VERSION: 10.0.18362.1 x86fre
THREAD_ATTRIBUTES: 
ADDITIONAL_DEBUG_TEXT:  Enable Pageheap/AutoVerifer ; Followup set based on attribute [Is_ChosenCrashFollowupThread] from Frame:[0] on thread:[PSEUDO_THREAD]
FAULTING_THREAD:  0005ef10
THREAD_SHA1_HASH_MOD_FUNC:  5d531e271dfb1ef7af4984c7ee0dd671c07337f5
THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  d858fa5fb04738fbbbbb9e4df89e26d53dc74794
OS_LOCALE:  ENU
BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_READ_HEAP_CORRUPTION
PRIMARY_PROBLEM_CLASS:  APPLICATION_FAULT

PROBLEM_CLASSES: 

    ID:     [0n262]
    Type:   [HEAP_CORRUPTION]
    Class:  Primary
    Scope:  DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
            BUCKET_ID
    Name:   Add
    Data:   Omit
    PID:    [0x6201c]
    TID:    [0x5ef10]
    Frame:  [0] : ntdll!RtlpAllocateHeap

    ID:     [0n262]
    Type:   [HEAP_CORRUPTION]
    Class:  Primary
    Scope:  BUCKET_ID
    Name:   Add
    Data:   Omit
    PID:    [0x6201c]
    TID:    [0x5ef10]
    Frame:  [0] : ntdll!RtlpAllocateHeap

    ID:     [0n313]
    Type:   [@ACCESS_VIOLATION]
    Class:  Addendum
    Scope:  BUCKET_ID
    Name:   Omit
    Data:   Omit
    PID:    [Unspecified]
    TID:    [0x5ef10]
    Frame:  [0] : ntdll!RtlpAllocateHeap

    ID:     [0n285]
    Type:   [INVALID_POINTER_READ]
    Class:  Primary
    Scope:  BUCKET_ID
    Name:   Add
    Data:   Omit
    PID:    [Unspecified]
    TID:    [0x5ef10]
    Frame:  [0] : ntdll!RtlpAllocateHeap

LAST_CONTROL_TRANSFER:  from 00007ffa46d69725 to 00007ffa46d6cb7a

STACK_TEXT:  
00000000`00000000 00000000`00000000 heap_corruption!javaw.exe+0x0
THREAD_SHA1_HASH_MOD:  ca4e26064d24ef7512d2e94de5a93c38dbe82fe9
SYMBOL_STACK_INDEX:  0
SYMBOL_NAME:  heap_corruption!javaw.exe
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: heap_corruption
IMAGE_NAME:  heap_corruption
DEBUG_FLR_IMAGE_TIMESTAMP:  0
STACK_COMMAND:  ** Pseudo Context ** ManagedPseudo ** Value: a3807e8 ** ; kb
FAILURE_BUCKET_ID:  HEAP_CORRUPTION_c0000005_heap_corruption!javaw.exe
BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_READ_HEAP_CORRUPTION_heap_corruption!javaw.exe
FAILURE_EXCEPTION_CODE:  c0000005
FAILURE_IMAGE_NAME:  heap_corruption
BUCKET_ID_IMAGE_STR:  heap_corruption
FAILURE_MODULE_NAME:  heap_corruption
BUCKET_ID_MODULE_STR:  heap_corruption
FAILURE_FUNCTION_NAME:  javaw.exe
BUCKET_ID_FUNCTION_STR:  javaw.exe
BUCKET_ID_OFFSET:  0
BUCKET_ID_MODTIMEDATESTAMP:  0
BUCKET_ID_MODCHECKSUM:  0
BUCKET_ID_MODVER_STR:  0.0.0.0
BUCKET_ID_PREFIX_STR:  APPLICATION_FAULT_INVALID_POINTER_READ_
FAILURE_PROBLEM_CLASS:  APPLICATION_FAULT
FAILURE_SYMBOL_NAME:  heap_corruption!javaw.exe
WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/javaw.exe/8.0.2210.11/5d1dea24/ntdll.dll/10.0.17134.799/7f828745/c0000005/0001cb7a.htm?Retriage=1
TARGET_TIME:  2019-12-02T10:16:32.000Z
OSBUILD:  17134
OSSERVICEPACK:  753
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK:  256
PRODUCT_TYPE:  1
OSPLATFORM_TYPE:  x64
OSNAME:  Windows 10
OSEDITION:  Windows 10 WinNt SingleUserTS
USER_LCID:  0
OSBUILD_TIMESTAMP:  unknown_date
BUILDDATESTAMP_STR:  180410-1804
BUILDLAB_STR:  rs4_release
BUILDOSVER_STR:  10.0.17134.1.amd64fre.rs4_release.180410-1804
ANALYSIS_SESSION_ELAPSED_TIME:  307a
ANALYSIS_SOURCE:  UM
FAILURE_ID_HASH_STRING:  um:heap_corruption_c0000005_heap_corruption!javaw.exe
FAILURE_ID_HASH:  {ddc2b378-b1e1-2aec-adc8-f11b7a5773a9}

Any help in fix/debug will be highly appreciated.

I got the solution of to prevent above heap corruption by calling NativeLibrary methods of PropertyHook in another thread. Somehow by calling NativeLibrary methods in different thread heap is not getting corrupted and sub-sequently JVM is not being crashed.