Calling C++ from JIT code (calling convention)

0

I'm programming a JIT compiler, that has to call c++ functions from the compiled asm code.

I'm working on Windows x64 only so far and implemented the calling convention from microsoft. (https://msdn.microsoft.com/en-us/library/ms235286.aspx)

It works, until it doesn't.

If the called function is simple and only calls functions that are simple (uses a C like subset), it seems to always work. If the called function calls other functions, e.g. "std::cout", it works if it's compiled for Debug, but not for release. If the called function calls some VULKAN functions, it doesn't work at all.

Example of the code the jit compiler generates:

sub  %10 r64(4),  imm8(48)
mov  %1 r64(0),  %6 r64(1)
mov  %2 r64(2),  m64[%1]
mov  %3 r32(1),  imm32(5)
mov  ST: m64 +40,  %2 r64(2)
call  m64 <call PrintInt()>
mov  %2 r64(2),  ST: m64 +40
mov  m32[%2] +16,  %4 r32(0)
mov  %5 r32(0),  imm32(-2)
add  %10 r64(4),  imm8(48)
ret  

Object (dump into https://www.onlinedisassembler.com/odaweb/ with x86-64):

48 83 EC 30 48 89 C8 48 8B 10 B9 05 00 00 00 48 89 54 24 28 FF 15 D6 FF FF FF 48 8B 54 24 28 89 42 10 B8 FE FF FF FF 48 83 C4 30 C3

The function called is as simple as (fails in release):

int PrintInt(int nr) {
std::cout << nr << std::endl;   // error
return nr;
}

The error is: "Exception thrown at 0x000007FEE3693D39 (msvcp140.dll) in test.exe: 0xC0000005: Access violation reading location 0x0000000000000000."

Any idea what I'm missing?

Edit the disassembly with my comments

sub rsp, 0x30               // subtract 48 bytes from stack pointer to get 6 slots of space
mov rax, rcx                // rcx is a pointer parameter
mov rdx, QWORD PTR [rax]    // load pointer from [rax]
mov ecx, 0x5                // imm 5 into ecx for function call
mov QWORD PTR [rsp+0x28], rdx   // save rdx for after function call on the stack
call QWORD PTR [..]         // function call
mov rdx, QWORD PTR [rsp+0x28]   // get rdx back from stack
mov DWORD PTR [rdx+0x10], eax   // save result
mov eax, 0xfffffffe             // -2 in eax (return value)
add rsp, 0x30               // restore stack ptr (free)
ret

STACK:

.... start
[XXX] used to save rdx
[   ] unused
[ s ]
[ s ]
[ s ]
[ s ]  shadow space
.... call

I'm reserving 6 slots (8 byte) so I get 16byte at the call alignment.

Thanks for any help.

c++
assembly
64-bit
jit
asked on Stack Overflow Sep 24, 2017 by Dany Bittel • edited Sep 24, 2017 by Dany Bittel

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0