LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 06-27-2013, 09:06 AM   #1
prushik
Member
 
Registered: Mar 2009
Location: Pennsylvania
Distribution: Lubuntu, BetterLinux
Posts: 370

Rep: Reputation: 29
Stumped while learning ASM. I know what to do to make it work, but can't grasp why.


Hi everybody.
I am learning x86_64 Linux assembly language. I got through the basics, making syscalls, comparisons and conditional jumps, accessing memory, registers, all that stuff. However, now I am trying to learn to write programs which use external libraries. Calling other functions is no problem, but when another piece of code is calling my function, that is where I am confused.
According to the amd64 ABI and Linux calling convention, I need to save some specific registers including RBP and RSP, which are the stack base pointer and stack pointer.
But if I don't use the stack, and all the functions that I call are responsible to save those registers correctly, shouldn't everything work if I never push or pop anything or modify those registers?
Even if something goes wrong, I think it would segfault on the ret instruction, but that is not what happens, my program segfaults at the first function call instead. I can't understand why that happens.

Also, I can get everything to run properly by simply pushing RBP onto the stack at the beginning of the function and popping it off at the end. So does the segfault have something to do with the state of the stack?

I have been searching for the answer, and some resources come close, but nothing explains why that needs to be done.
The best one I found so far was this: http://cs.nyu.edu/courses/fall11/CSC.../x64-intro.pdf
It almost explains it, but doesn't quite get there.

Others show what needs to be done, but don't explain why, like this one: http://vikaskumar.org/amd64/sample.htm

Also, compiling a small C program and examining the resultant ASM code shows the same thing: PUSH RBP and then set RBP to RSP. Which I think I understand, but I can't figure out why it segfaults before the RET instruction if I don't do that.
 
Old 06-27-2013, 09:16 AM   #2
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,139

Rep: Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127
Quote:
Originally Posted by prushik View Post
But if I don't use the stack, and all the functions that I call are responsible to save those registers correctly, shouldn't everything work if I never push or pop anything or modify those registers?
Yes. If you never modify those registers (even if you do use the stack) then you don't need to save/restore them. If other functions you call modify those registers, it is their responsibility to restore them.

Quote:
my program segfaults at the first function call instead. I can't understand why that happens.
At the function call? Or do you mean somewhere inside that function?

Quote:
Also, I can get everything to run properly by simply pushing RBP onto the stack at the beginning of the function and popping it off at the end. So does the segfault have something to do with the state of the stack?
Sounds like stack alignment. In x86_64, you shouldn't simply call another function on entry to your function. The call into your function misaligned the stack by 8 bytes. You need to push another 8 bytes (or any odd multiple of 8 bytes) onto the stack before calling another function.

Quote:
I can't figure out why it segfaults before the RET instruction if I don't do that.
Very few functions would crash due to stack misalignment. Some would just run slower. Others wouldn't care at all. But I don't have a guess other than stack misalignment based on your description. If stack misalignment is the problem, it should crash deep inside the other function you called on some operation that depends on stack alignment.

You "fixed" the problem by saving and restoring rbp, creating what I think is an illusion that you needed to save/restore rbp. Try pushing/popping r11 instead of rbp. We know nothing cares about saving/restoring r11, so when that also "fixes" the problem, you have pretty much confirmed it was stack alignment.

Last edited by johnsfine; 06-27-2013 at 09:24 AM.
 
1 members found this post helpful.
Old 06-27-2013, 11:13 AM   #3
prushik
Member
 
Registered: Mar 2009
Location: Pennsylvania
Distribution: Lubuntu, BetterLinux
Posts: 370

Original Poster
Rep: Reputation: 29
Quote:
Originally Posted by johnsfine View Post
Yes. If you never modify those registers (even if you do use the stack) then you don't need to save/restore them. If other functions you call modify those registers, it is their responsibility to restore them.
Ok, sounds good.

Quote:
Originally Posted by johnsfine View Post
At the function call? Or do you mean somewhere inside that function?
Probably inside the function.


Quote:
Originally Posted by johnsfine View Post
Sounds like stack alignment. In x86_64, you shouldn't simply call another function on entry to your function. The call into your function misaligned the stack by 8 bytes. You need to push another 8 bytes (or any odd multiple of 8 bytes) onto the stack before calling another function.
Stack alignment. Good call. You are sharp my friend.


Quote:
Originally Posted by johnsfine View Post
Very few functions would crash due to stack misalignment. Some would just run slower. Others wouldn't care at all. But I don't have a guess other than stack misalignment based on your description. If stack misalignment is the problem, it should crash deep inside the other function you called on some operation that depends on stack alignment.
After reading your comment, I did some research and found that some SSE instructions will segfault if the stack is misaligned. I am calling SDL, which is built with SSE and SSE2 instructions enabled. So that sounds plausible. Awesome.


Quote:
Originally Posted by johnsfine View Post
You "fixed" the problem by saving and restoring rbp, creating what I think is an illusion that you needed to save/restore rbp. Try pushing/popping r11 instead of rbp. We know nothing cares about saving/restoring r11, so when that also "fixes" the problem, you have pretty much confirmed it was stack alignment.
I think you are right. I did try your suggestion of using r11 instead and it worked fine, so think your answer is correct. Thank you. I understand much better now.

However, this seems a bit awkward now, I need to align the stack by pushing some random data onto, even though I don't need the data. Is there a more graceful way to align the stack?
 
Old 06-27-2013, 12:31 PM   #4
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by prushik View Post
...Is there a more graceful way to align the stack?
Explicitly modify stack pointer.
 
Old 06-27-2013, 01:12 PM   #5
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,139

Rep: Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127
Personally I prefer push/pop for stack alignment (preferably with a comment saying just for stack alignment).

Maybe you think it is more graceful to use
Code:
  sub $8,%rsp
  ...
  add $8,%rsp
That is a valid opinion if you like.
Sometimes that code is also faster than push/pop but very unlikely to make a difference that matters and there are many factors (instruction alignment etc.) that might overwhelm the superficial time difference and even reverse which version is slightly faster.
 
1 members found this post helpful.
Old 06-27-2013, 01:14 PM   #6
prushik
Member
 
Registered: Mar 2009
Location: Pennsylvania
Distribution: Lubuntu, BetterLinux
Posts: 370

Original Poster
Rep: Reputation: 29
Quote:
Originally Posted by Sergei Steshenko View Post
Explicitly modify stack pointer.
Sounds good, I can just do something like:
sub rsp,8

If I do that, then I have to restore it also before I return, should I use:
add rsp,8

If I have to restore rsp at the end, isn't it just as graceful to save rsp on the stack and pop it off at the end?
 
Old 06-27-2013, 04:12 PM   #7
JohnGraham
Member
 
Registered: Oct 2009
Posts: 467

Rep: Reputation: 138Reputation: 138
I haven't used x86 assembly in a while, but you don't align a stack by adding a constant value to it - what if it's already aligned? You usually OR the stack pointer - e.g. OR it with 0xfffffff0 to align a 32 bit stack pointer by 16 bytes. And IIRC, you save %esp as %ebp on function entry for x86, so you shouldn't need undo what you've done at the end (again, my x86 assembler is rusty...).

Last edited by JohnGraham; 06-27-2013 at 04:18 PM.
 
Old 06-27-2013, 05:12 PM   #8
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,139

Rep: Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127
Quote:
Originally Posted by JohnGraham View Post
I haven't used x86 assembly in a while, but you don't align a stack by adding a constant value to it - what if it's already aligned? You usually OR the stack pointer - e.g. OR it with 0xfffffff0 to align a 32 bit stack pointer by 16 bytes. And IIRC, you save %esp as %ebp on function entry for x86, so you shouldn't need undo what you've done at the end (again, my x86 assembler is rusty...).
Your discussion (excepting some details) applies when you want higher alignment than is standard for the environment in which your code will be run.

The situation of this thread was standard x86_64 ABI, in which the stack is 16 byte aligned before each call, so on entry to a function it is assumed to be 8 bytes off from 16 byte aligned. So you can subtract 8 from rsp in order to restore 16 byte aligned.
 
Old 06-27-2013, 05:29 PM   #9
mina86
Member
 
Registered: Aug 2008
Distribution: Slackware
Posts: 412

Rep: Reputation: 172Reputation: 172
Quote:
Originally Posted by JohnGraham View Post
I haven't used x86 assembly in a while, but you don't align a stack by adding a constant value to it - what if it's already aligned? You usually OR the stack pointer - e.g. OR it with 0xfffffff0 to align a 32 bit stack pointer by 16 bytes.
And by OR you of course meant AND.
 
Old 06-27-2013, 05:36 PM   #10
mina86
Member
 
Registered: Aug 2008
Distribution: Slackware
Posts: 412

Rep: Reputation: 172Reputation: 172
Quote:
Originally Posted by prushik View Post
If I have to restore rsp at the end, isn't it just as graceful to save rsp on the stack and pop it off at the end?
SUB and ADD is faster then PUSH and POP, and besides, even though PUSH RSP and POP RSP will work correctly, they look a bit weird (at least to me) since PUSH and POP respectively decrement and increment RSP register.
 
Old 06-28-2013, 02:33 AM   #11
prushik
Member
 
Registered: Mar 2009
Location: Pennsylvania
Distribution: Lubuntu, BetterLinux
Posts: 370

Original Poster
Rep: Reputation: 29
Thanks everybody. I am marking this thread as solved since you guys answered my question. Thanks.
 
Old 06-29-2013, 05:50 AM   #12
JohnGraham
Member
 
Registered: Oct 2009
Posts: 467

Rep: Reputation: 138Reputation: 138
Quote:
Your discussion (excepting some details) applies when you want higher alignment than is standard for the environment in which your code will be run.

The situation of this thread was standard x86_64 ABI, in which the stack is 16 byte aligned before each call, so on entry to a function it is assumed to be 8 bytes off from 16 byte aligned. So you can subtract 8 from rsp in order to restore 16 byte aligned.
Ah, I see - fair enough.


Quote:
And by OR you of course meant AND.
I did indeed - silly me :-P
 
  


Reply

Tags
amd64, asm, registers, stack, x86_64


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Ethernet will not work... Have tried everything. I am officially stumped. LiquidCougar Slackware 3 12-15-2012 07:04 PM
inline asm, 64-bit asm, intercepting segfaults, etc. rainbowsally Programming 0 02-04-2012 06:10 PM
please help! How do we make our Linux RedHat server recognize the new LUNs (ASM)? kmagitz Linux - Newbie 3 08-25-2010 07:17 AM
my NIC died , cant get new ones to work. stumped. modprobe Red Hat 7 07-23-2008 02:19 AM
ASM x32 vs ASM x64 Tegramon Programming 3 02-27-2008 03:26 PM


All times are GMT -5. The time now is 01:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration