How to fix "Unhandled exception" error in assembly?

1

I've written a function that determines if a value is prime or not prime. But when I return from the function, it comes up with an error.
The error message is

Unhandled exception at 0x00000001 in Project.exe: 0xC0000005: Access violation executing location 0x00000001.

This function should return eax.

        push ebp
        mov ebp, esp
        mov eax, [ebp+8]                ; store input value
        mov ecx, [ebp+8]                ; store input value in counter
        sub esp, 4                      ; local variable 
        sub ecx, 1                      ; avoid compare with itself
        cmp eax, 3                      ; compare with 1, 2, 3
        jbe Prime

    L1:
        cmp ecx, 3                      ; when count=3 to stop
        je NotP
        mov edx, 0                      ; clear edx to save remainder
        mov [esp-4], eax                ; save input value
        div ecx                         ; divide number
        cmp edx, 0                      ; check remainder
        je NotP                         ; if remainder=0 then not prime
        jmp Prime
        loop L1
    NotP:
        mov eax, 0
        push eax                        ; if delete this ilne still come up error
        pop ebp
        ret
    Prime:
        mov eax, 1                      
        push eax                        ; if delete this ilne still come up error
        pop ebp
        ret
    isPrime  endp
function
assembly
x86
primes
masm
asked on Stack Overflow May 7, 2019 by tong0929 • edited May 9, 2019 by Fifoernik

1 Answer

2
   mov [esp-4], eax                ; save input value

If you plan on using the local variable that you reserved room for, then you have to write:

    mov [esp], eax                ; save input value

or alternatively write:

    mov [ebp-4], eax              ; save input value

A correct prolog/epilog would be:

    push    ebp
    mov     ebp, esp
    sub     esp, 4                 ; local variable 
    mov     eax, [ebp+8]           ; store input value

    ...

NotP:
    mov     eax, 0
    pop     ebp                    ; remove local variable
    pop     ebp
    ret
Prime:
    mov     eax, 1                      
    pop     ebp                    ; remove local variable
    pop     ebp
    ret
isPrime  endp

    cmp edx, 0                      ; check remainder
    je NotP                         ; if remainder=0 then not prime
    jmp Prime
    loop L1

Finding the remainder not zero is not enough to conclude that the number is prime! More tests are needed. For now, that loop L1 instruction is never executed.
e.g. To test 15, your first division does 15 / 14 which yields a non-zero remainder but 15 isn't a prime number.

L1:
    cmp ecx, 3                      ; when count=3 to stop
    je NotP

The top of the loop can't be correct either! Consider testing the number 7.
First division is 7 / 6 and has a remainder so the loop has to continue
Second division is 7 / 5 and has a remainder so the loop has to continue
Third division is 7 / 4 and has a remainder so the loop has to continue
You don't try any more divisions and conclude "not prime", yet 7 is definitively a prime number.

answered on Stack Overflow May 7, 2019 by Sep Roland • edited May 7, 2019 by Sep Roland

User contributions licensed under CC BY-SA 3.0