Why this assemble code occur Segmentation Fault?

0

Why this code gives segmentation fault? (It coded by Intel Assemble syntax)

The result is like this.

return value : 80






[Dump]
eax : 0x00000012
ebx : 0x00000004
ecx : 0x00000400
edx : 0x4010a980
Segmentation fault (core dumped)

I think I have enough code to prevent Segmentation Fault. Every function has prologue and epilogue to maintain stack memory.

But, It occur Segmentation Fault error.

[Additional Note] If I remove the code 'mov ebx, 4', then segmentation fault error had been removed. (But the result is not fit to my intention)

extern printf

segment .data
dumpmsg db      10,10,10,10,10,10,'[Dump]',10,'eax : 0x%0.8x',10,'ebx : 0x%0.8x',10,'ecx : 0x%0.8x',10,'edx : 0x%0.8x',10,00
msg     db      'return value : %d', 10, 00

segment .bss

segment .text
global  main
main:
        push    ebp
        mov     ebp,    esp

        push    20
        call    pig
        add     esp,    4

        push    eax
        push    msg
        call    printf

        call    print_x
        leave
        ret

pig:
        push    ebp
        mov     ebp,    esp

        mov     eax,    [ebp+8]
        mov     ebx,    4
        mul     ebx

        leave
        ret

print_x:
        push    ebp
        mov     ebp,    esp

        push    edx
        push    ecx
        push    ebx
        push    eax
        push    dumpmsg
        call    printf

        leave
        ret
assembly
segmentation-fault
nasm
asked on Stack Overflow Jul 22, 2017 by YouHoGeon • edited Jul 22, 2017 by Michael Petch

1 Answer

3

If I remove the code 'mov ebx, 4', then segmentation fault error had been removed. (But the result is not fit to my intention)

The EBX register is callee-save in nearly all x86 calling conventions. That means that a leaf function cannot clobber its value. If it needs to use EBX, it must save its original value at the top of the function and restore it at the end. This is usually accomplished with PUSH+POP instructions to save the register's value on the stack.

Your pig function is clobbering EBX with the mov ebx, 4 instruction, but it is not taking care to save and restore its original value. This violates the calling convention, which matters because you are interoperating with C code.

The fix is simple:

pig:
    push    ebp
    mov     ebp,    esp
    push    ebx

    mov     eax,    [ebp+12]
    mov     ebx,    4
    mul     ebx

    pop     ebx
    leave
    ret

Or, just use the ECX register to hold the divisor, since this one is caller-save (and thus free to clobber):

pig:
    push    ebp
    mov     ebp,    esp

    mov     eax,    [ebp+12]
    mov     ecx,    4
    mul     ecx

    leave
    ret

Every function has prologue and epilogue to maintain stack memory.

This really isn't necessary. The pig function doesn't use the stack at all, except to retrieve a parameter, so you could write it simply as:

pig:
    mov     eax,    [esp+4]
    mov     ecx,    4
    mul     ecx
    ret

The use of the base pointer (EBP) could be similarly elided from other functions, even those that use the stack, by using the stack pointer (ESP) directly. This is the same optimization that compilers will make to free up EBP as an additional register and save the prologue/epilogue bloat. But if you are more comfortable doing it this way, then it doesn't hurt anything except performance.

What is essential is that you follow the calling convention. The EAX, EDX, and ECX registers are free to clobber inside of a leaf function; all the rest of them need to be explicitly saved and restored to their original values. When you allocate space on the stack, you need to make sure to free it. How you do so is flexible, whether it's with sub esp, xxadd esp, xx or the prologue...epilogue you have.

answered on Stack Overflow Jul 22, 2017 by Cody Gray • edited Jul 22, 2017 by Cody Gray

User contributions licensed under CC BY-SA 3.0