Recently I’ve finished the Practical Malware Analysis book and I’ve wanted to familiarise myself a bit more with the Win32 API. After spending a good amount of time on setting up Visual Studio C++ for MASM (Microsoft Macro Assembler) I wanted to stab myself in the eye with a rusty fork due to the overload of visual clutter. Alas, running plain MASM on Windows 10 seems to be a no-go these days. Folks have been using Dosbox to run an (even) older version of MASM but instead I went down another rabbit hole entirely.

Requirements⌗

Thanks to Paul Irofti we have a working port of the mingw toolchain in OpenBSD ports so I figured I’d give that a shot to link code produced by nasm. While this post was written on OpenBSD, it does of course apply to all non-Windows systems with nasm and mingw.

First install the required packages:

pkg_add mingw nasm

Additionally I tend to install py3-numpy to make the conversion from hex to signed decimal numbers a little bit easier (along with other common operations). It’s probably overkill but this works fine:

import numpy
print(numpy.int32(0xfffffff5)) # -11
print(0xfffffff5)              # 4294967285

Hello.nasm⌗

I wanted to demonstrate a trivial example which writes “Hello World” to the console, followed by a popup with a greeting:

global _main
extern  _ExitProcess@4
extern  _GetStdHandle@4
extern  _MessageBoxA@16
extern  _WriteFile@20

section .text

_main:
	mov     ebp, esp
	; Allocate space for lpNumberOfBytesWritten (DWORD)
	sub     esp, 4
	
	push    -11             ; STD_OUTPUT_HANDLE
	call    _GetStdHandle@4 ; returns a HANDLE
	mov     ebx, eax
	
	push    0               ; lpOverlapped
	lea     eax, [ebp-4]
	push    eax             ; lpNumberOfBytesWritten
	push    msg_len         ; nNumberOfBytesToWrite
	push    msg             ; lpBuffer
	push    ebx             ; hFile
	call    _WriteFile@20
	
	push    0x40            ; uType (MB_ICONINFORMATION)
	push    msg_caption     ; lpCaption
	push    msg_text        ; lpText
	push    0               ; hWnd
	call    _MessageBoxA@16
	
	push    eax             ; uExitCode
	call    _ExitProcess@4

section .data

msg:
        db      'Hello, World', 10, 0
msg_len: equ $-msg

msg_caption:
        db 'MessageBox', 0
msg_text:
        db 'Hello World!', 0

It can be assembled and linked with:

tau:2044 win32_nasm % nasm -f win32 -o hello.o hello.nasm
tau:2045 win32_nasm % i386-mingw32-gcc -o hello.exe hello.o

As even such a simple binary is fairly large (421kb) you’d probably want to strip it and suddenly only an 8gb hello.exe remains.

Not an unimportant detail, it runs too! At least on Windows XP SP3 and Windows 10 (using WoW64):

win32-nasm-hello.png

Win32 API and __stdcall details⌗

Note that I’m not using the printf() function from the C library provided by GCC; instead I’m using the Win32 API directly. One thing that clearly stands out here are the names of the imported functions, such as _WriteFile@20. This is in part due to assembling and linking on a non-native platform and in part it’s the __stdcall calling convention at work. Let me try to explain.

If we were to be building this natively on Windows we could have written:

extern ExitProcess
import ExitProcess kernel32.dll
; etc

And adjusted the nasm command to use -f obj to produce a raw object file, however gcc/ld cannot cope with that. On Windows one could use alink instead but that’s not an option here. Have a look at this post if you are working a Windows system instead. So instead we have to explicitly comply with the name-decoration convention as set forth in the MSDN __stdcall documentation. This means prefixing the name of the function with an underscore, and concatenating this with @ and the number of bytes (in decimal) which make up the argument list.

You can the MSDN documentation to determine those function names:

Alternatively looking at the relevant library can help too to lookup the size:

tau:2056 lib % nm /usr/local/mingw32/lib/libkernel32.a |grep T\ _WriteFile
00000000 T _WriteFileVlm@20
00000000 T _WriteFileGather@20
00000000 T _WriteFileEx@20
00000000 T _WriteFile@20
tau:2057 lib %

Obviously this was a trivial example, but I hope it makes it easier to assemble and link assembly using the Win32 API in non-native environments.