CreateFileA in Windows API in NASM 64: incorrect parameter, but which one?

-1

I am creating a file using CreateFileA from the Windows API in NASM 64-bit (see https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea). With the following parameters, no file is created and it returns an error 87 ("the parameter is incorrect") from GetLastError (see https://docs.microsoft.com/en-us/windows/desktop/debug/system-error-codes--0-499-)

Here are the parameters:

rcx - lpFileName

;dwDesiredAccess

mov rdx,2 I chose FILE_WRITE_DATA from https://docs.microsoft.com/en-us/windows/desktop/FileIO/file-access-rights-constants

; dwShareMode

mov r8,0

According to https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea: If this parameter is zero and CreateFile succeeds, the file or device cannot be shared. According to https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea, the value should be zero for no sharing.

; lpSecurityAttributes

mov r9,const_inf ; (Pointer to null value dq 0xFFFFFFFF) OR mov r9,const_0 According to https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea: "If this parameter is NULL, the handle returned by CreateFile cannot be inherited by any child processes the application may create and the file or device associated with the returned handle gets a default security descriptor."

sub rsp,24 ; stack space

; dwCreationDisposition

mov rax,2 (CREATE_ALWAYS)

mov [rsp+16],rax

; dwFlagsAndAttribute

mov rax,128

mov [rsp+8],rax

The value 128 is from https://docs.microsoft.com/en-us/windows/desktop/FileIO/file-attribute-constants

; hTemplateFile

mov rax,[const_inf]

mov [rsp+0],rax


Here is the full file creation code:

CreateAuditFile:
push r10
mov r10,rax ; Core #
mov rdi,FileHandles
mov rbx,[rdi+r10]
cmp rbx,0 ; has file been created
jne file_next
mov rcx,FileName_1
mov rdx,2 ;dwDesiredAccess ;0x40000000
push r8
push r9
mov r8,0 ; dwShareMode
mov r9,const_0 ; lpSecurityAttributes
;OR:  mov r9,const_inf ; lpSecurityAttributes
; CREATE STACK SPACE FOR REMAINING PARAMETERS:
sub rsp,24
mov rax,2 ; dwCreationDisposition (CREATE_ALWAYS)
mov [rsp+16],rax
mov rax,128
mov [rsp+8],rax ; dwFlagsAndAttributes
mov rax,[const_inf]
mov [rsp+0],rax ; hTemplateFile
push r10
call CreateFileA
pop r10
mov rdi,FileHandles
call GetLastError
mov [rdi],rax
add rsp,24
pop r9
pop r8
pop r10
file_next:
ret

I have looked carefully at the parameter options, but the error message only says "invalid parameter." It doesn't say which parameter.

My question is: which parameter or parameters above is incorrect? Are the parameters on the stack passed correctly?

Thanks for any help.

windows
winapi
nasm
asked on Stack Overflow Apr 2, 2019 by RTC222

1 Answer

0

I solved this problem, and here is the solution. The stack handling on my original question was incorrect. The right way to handle the stack is shown below.

The values for each of the parameters (such as DesiredAccess, ShareMode and Security Attributes) may be different depending on the specific needs of the project, but the parameters are passed as in the code below:

CreateAuditFile:
mov rcx,FileName_1
sub rsp,56  ; 38h
xor eax,eax
mov qword [rsp+48],rax ; 30h
mov eax,80
mov dword [rsp+40],eax ; 28h
mov eax,2
mov dword [rsp+32],eax ; 20h
xor r9,r9
xor r8d,r8d
mov edx,40000000
call CreateFileA
mov rdi,OutputFileHandle
mov [rdi+r15],rax
xor eax,eax
add rsp,56 ;38h
ret

Thanks very much to everyone who responded.

answered on Stack Overflow Apr 5, 2019 by RTC222 • edited Apr 5, 2019 by RTC222

User contributions licensed under CC BY-SA 3.0