LinuxQuestions.org - [SOLVED] call c function from assembly in 64 bit mode

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - call c function from assembly in 64 bit mode (https://www.linuxquestions.org/questions/programming-9/call-c-function-from-assembly-in-64-bit-mode-4175675562/)

call c function from assembly in 64 bit mode

Am having a problem with static calls from assembly code to a C function. I suppose the problem is with the different calling conventions between 32 bit and 64 bit compilers. In my case, my home computer is a 32 bit Linux, where everything is working fine. The server where i compile the same program is 64 bit BSD. Am using gcc to compile both the C and assembly code, on both machines.

My principal code is a C program with a number of functions that may be called from a variety of points, sometimes recursively. It also calls a couple of assembly language routines contained in their own source code files, compiled and linked statically.

At a couple of points, i need to call one of the C functions from within the assembly code. Weirdly, the call works fine most of the time, say around 70 or 80 percent of the time, but sometimes fails at the call with a Seg violation or signal 10 (ten), which tech support says is a bus error. I cannot use gdb or other interactive debugging tools, just have to drop a value into a global variable which i can then look at after catching the signal.

The code is far too complex to show here. In simplified form, it goes something like this: The C code in my_program.c:

Code:

int c_function(int);



int main (int argc, char *argv[]) {

int var1 var2, ret_c, ret_a;

ret_c = c_function(var1);

.

.

ret_a = asm_func(var2);

.

.

}



int c_function(int my_var) {

do something here

}

The assembly module (asm_func.S):

Code:

.text

.globl asm_func

  .type    asm_func, @function

asm_func:

.

.

'prologue' and other assembly stuff

. 

.

pushq %rdi

movq a_var,%rdi

call c_function          /* SOMEtimes crashes right here */

popq %rdi

.

.

'epilogue' stuff

ret

The compile instructions in makefile:

Code:

asm_func.o : asm_func.S ;

    gcc -c -Wall asm_func.S -o asm_func.o



my_program : my_program.c asm_func.o ;

    gcc -s my_program.c asm_func.o -o my_program

I may try to write a simple routine (like the above) to try to make the problem occur in such a way as to give better details. Meanwhile, anybody that has experience in doing this sort of thing in 64 bit archtecture, i'd welcome a tip or two. Thanks

Quote:

Originally Posted by dogpatch (Post 6125180)

Code:

int c_function(int);

Instead of relying on int that could have different size on different processor architectures, I'd use C99 explicit integer types intn_t here.

Thanks, shruggy, but that isn't the problem. The sample code above is a very simplified picture of what my code is actually doing. In reality, i have a number of functions that accept different passed arguments or no arguments, and that produce different sized return values, or no return value. The common factor is that they crash at the call from assembly to a C function. They crash precisely at the call instruction, and intermittently, not all the time. I haven't yet figured out what is different about the failed calls.

Am beginning to suspect that this is a nasty random corruption of memory or of the stack pointer, or something like that, nothing to do with the call itself.

Have worked around the worst case for now, but will have to try at some point to make this happen with some simpler code.

You seem to be aware that the 32-bit and 64-bit x86 calling conventions are completely different.

You have to write completely different assembly code for 32-bits and 64-bits. That the 32-bit code works tells you nothing about the 64-bit code.

You should compile the C function into assembly to study it. Check that the parameters are being passed correctly and that the C function is not clobbering registers used by the caller. Make sure that the upper 32-bits are being preserved. It is easy to write %eax instead of %rax by mistake.

Assembly code should be avoided because there are so many pitfalls. If you must write assembly code, put it inline in a C function so that fewer things can go wrong.
Ed

Yes, i am well aware of the difference in calling conventions. The 64-bit calling conventions are actually much simpler. Am also aware of the pitfalls of inter-language coding. But converting the assembly routine to C or in-line assembly is out of the question. Won't go into details right now.

Yes, I have compiled the C function into assembly, and have studied both how the function is coded, and how the function is called from other parts of the C program.

I should emphasize again that the code works in 64 bits as well as in 32 bits. At least, most of the time. Nothing appears to be clobbered at either end. The call instruction itself is where the fault is occuring. This could be because the stack register is somehow corrupted. Or, speaking of the stack, another thought has just occurred to me:

Do i vaguely remember reading somewhere that the stack pointer should be aligned to a 16-byte boundary for 64-bit calls? Can someone at LQ confirm or deny this? This would be something that the compiler would just do correctly as a matter of course within the C code before any call, or at least before an external call. It might also explain why my problem is intermittent, depending upon what the rsp value happens to be at the time the call instruction is executed. Would a poorly aligned stack pointer at a call instruction result in a seg fault or bus error?

At any rate, that may be what i look at next. But not right now. Have been up all night. Maybe i'll check back here again later, much later.

Yes, the stack pointer should be aligned to 16-bytes due to the XMM registers. On x86, misalignment generally results in lower performance. I would align the stack pointer to satisfy the ABI.

I presume that you have inherited a poorly-designed mess of C and assembly. I hope you are being paid well.

ETA: Double-check any use of 32-bit registers in 64-bit code. Writing to a 32-bit register zeros the upper 32 bits.
Ed

Aligning the stack pointer to a 16 byte boundary right before the call seems to have fixed the problem. Thanks, EdGr; have given you a reputation point.

(this code isn't really such a mess, but am still a 64-bit noobie, and didn't know about the stack alignment requirement)