I am trying to write code that takes input from the user and displays the factorial, in assembly x64. Every time I run the code it gives me Unhandled exception at 0x00007FF64B910B11 in finalProjectAssembly.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x00000019F1603FF8)., but I do not understand how the stack has overflowed. What am I missing?
INCLUDELIB libcmt.lib
INCLUDELIB legacy_stdio_definitions.lib
EXTERN printf:PROC
EXTERN scanf:PROC
.DATA
prompt BYTE "Enter a number: ", 0
inFmt BYTE "%d", 0
prompt3 BYTE "The factorial is: %d",10,0
cont QWORD ?
disp BYTE "You Entered %d", 10,0
num QWORD ?
num2 REAL8 1.5
.CODE
main PROC C
sub rsp, 24
lea rcx, prompt
call printf
lea rdx, num
lea rcx, inFmt
call scanf
push num
mov rdx,num
lea rcx, disp
call printf
call factorial
mov rdx, rax
lea rcx, prompt3
call printf
lea rcx, prompt2
call printf
lea rdx, cont
lea rcx, inFmt
call scanf
add rsp,24
mov rax,0
ret
main ENDP
factorial PROC
push rbp
mov rbp,rsp
mov rax, [rbp + 16]
cmp rax, 1
jle quit
dec rax
push rax
call factorial
mov rbx, [rbp+16]
imul rbx
quit:
mov rsp, rbp
pop rbp
ret
factorial ENDP
If you call foreign functions (printf
,scanf
) you must obey their calling conventions. The appropriate Microsoft x64 software convention determines that you have to reserve at leat space for four qwords (=32 bytes) in the stack before calling the function. The function can freely use that "shadow space". Furthermore, ESP have to be aligned to an address divisible by 16.
Change
sub rsp, 24
to
sub rsp, 32
and spl, -16 ; Align to 16
You don't need the add rsp,24
at the end of the main
function. The correct prolog (push rbp; mov rsp, rbp
) and the correct epilog (leave
) will be inserted automatically by the assembler ML64.
The shadow space has to be straight in front of the function's return address pushed by the call
instruction. Move the push num
close to call factorial
and don't forget to clean up the stack:
push num
call factorial
add rsp, 8
Don't forget to clean up the stack:
push rax
call factorial
add rsp, 8
I didn't check the format specifier %d
as recommended by David Wohlferd. The program basically works with the changes above.
User contributions licensed under CC BY-SA 3.0