[SOLVED] Passing args from C to NASM

WarTurkey · 01-30-2012, 06:55 PM

Hey, I've been working on a program for a little while, I call it Inegma (I posted it on SourceForge a while ago, eh what the heck.) I'm not really good at programming, and have stepped away from it for a while. I came back and I happened to encrypt a file with my program and I forgot the code to decrypt it! I'm trying to write a program to brute force my own encryption, but I keep getting errors when I try to pass an argument to my assembly function. In my original program, I had no problems, but it's not working in this implementation:
In C, I have it written like:

Code:

char encode(char, char);
int	i,	j;
char *test_string;
...
crypto_array[i][j]=encode(j, test_string[i]);

And then the assembly file has two versions. One for 64-bit, because the arguments are passed in registers there, and another for 32-bit, because it used the stack there, but what I do is:

Code:

mov	cl,al	;because eax contains "j"
mov	al,bl	;and ebx contains "test_string[i]"

Now, I know I'm just grabbing the least significant bits of j, but because the values are only ever between 0x00 and 0x0F I thought that wasn't an issue, but maybe I'm wrong.. And I have tried both defining j as an unsigned char and casting it as one.

johnsfine · 01-31-2012, 11:54 AM

Where do those two lines of asm code fit in?
Even in 64 bit where arguments are passed in registers, the first two arguments are not passed in rax and rbx. By x86-64 standard, the first six arguments are passed in rdi, rsi, rdx, rcx, r8 and r9 (Windows doesn't follow that standard).

Quote:

Originally Posted by WarTurkey

I keep getting errors when I try to pass an argument to my assembly function.

You mean compile time errors in the C code? Or in the asm code? Or a runtime error message? Or just an incorrect result? Or what?

WarTurkey · 01-31-2012, 12:15 PM

As I mentioned in my edit, my question is really bad. I'm going to post everything when I get home, but no, compile time, there are no errors, and it's strange that the arguments are passed that way, and that yet my original program works. I'll have it all posted when I get home.

I was actually trying to see how to delete this thread and I would post a revised form of my question later, but then I couldn't find any option to delete it.

WarTurkey · 01-31-2012, 02:37 PM

You're right, I need to move them to edi, esi (from disassembling my program), but now I don't understand how my initial program works!

johnsfine · 01-31-2012, 02:47 PM

The compiled C code will typically compute a function argument into the wrong register and then copy it to the correct register, so at the moment the asm code is called the same value is in two different registers.

But the tiniest change to the C code, the compiler version or the compile command line can change which registers are used to compute the argument values.

So I assume in your earlier version the argument values happened to be in rax and rbx as a random compilation effect as well being in rdi and rsi as a result of the x86-64 calling stantard.

WarTurkey · 01-31-2012, 04:19 PM

Ugh. Ok, this is the fourth time I've gone to type out a response. I'm getting myself completely lost. I found that I must have simply accessed the registers eax and ebx because their values were moved into edi and esi immediately before calling encode(). However, I adjusted the assembly source code to pull from esi and edi, and it seems to work for values from 0-7, but for values 8-F, it's supposed to invert the number.

It doesn't matter, the thread is essentially solved in that I was trying to grab from the wrong registers.