Hi all, I am an new member and this is my first post.
Hope I am posting at the correct forum.


I am using SLC 4.5 (Kernel 2.6.9-55.EL.cernsmp) on a 64-bit machine. I am doing programming in GNU C.

The thing that I am trying to do is this...

The C program forks a new process and execs an executable file. The newly forked process allows the parent (C program) to trace it by passing the PTRACE_TRACEME request in a ptrace call. Now the parent has to map a certain memory area into the child's address space. The parent process does this in the following manner:

1. Save the original registers contents of the child.
2. Write 0x80cd (INT 80 instruction) at the stack top and point the eip to this location.
3. Load the registers contents as
rax = __NR_mmap ; //System call No. This stores 0x9 in rax as specified in <asm/unistd.h>
rbx = 0x501000; //Start address of the area to be mapped.
rcx = 0x21000; //Size of the area to be mapped.
rdx = PROT_READ | PROT_WRITE | PROT_EXEC; //Permissions of the area to be mapped
rsi = MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS; // flags

During this time rip equals 0xBFFFF028
4. Parent process makes the child execute the current instruction i.e. INT 80 (using PTRACE_SINGLESTEP)
5. The parent restores the original register contents of the child.

But what really is happening is different. After step 4 the register contents are:

rax = 0xFFFFFFF2
rbx = 0x501000;
rcx = 0x21000;
rdx = 0x7 (i.e. same as earlier PROT_READ | PROT_WRITE | PROT_EXEC)
rsi = 0x32 (i.e. same as earlier MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS)

Here rip equal 0xBFFFF02A ( an increment of two bytes i.e. 0x80CD)

After Step 4 i.e. after execution of ptrace(PTRACE_SINGLESTEP,.....) by the parent the errno is 0. So this means that execution of ptrace was successful. The return value of mmap call (which is supposed to be in register rax) is 0xFFFFFFF2 (i.e. -14).

Since the return value is not MAP_FAILED, it should be assumed that mmap call was successful. But since the value is completely outside the address range to be mapped, it is difficult to know what happens during the mmap system call.

Does anybody have any idea that why such a value is being returned in the register rax???
Is it garbage value or some error code?? (although I saw the error codes but there was no macro corresponding to value -14)
Is it related to memory pointers???

And I must tell you that the above thing when implemented on a 32 bit machine with kernel version 2.4 works well (of course, there the registers are eax, ebx etc. in place of rax, rbx....)

What can be the changes in kernel which prevent in from working on a higher kernel version??

Thanks in advance for any help..........

can you please try to run it on an updated kernel and post results? (>=2.6.27).

it does seem odd, but alot of changes were made since 2.6.9 that might be relevant here./

waiting for your answer.

First of all thanks for the quick reply.

Well, it will be difficult to test on a 2.6.27 kernel version as I am working on a production system and finally the application is to be run on a similar (version 2.6.9) system.

My doubt is that why it could work on 2.4 kernel and not on a 2.6 kernel? Since Linux 2.6 also provides the facility of making system calls using the INT 80 instruction and the calling procedure as well as return value of mmap system call remains same. Although I am not sure whether the actual implementation between the mmap call and return has been modified.

Also, is there any other way by which a parent process can map a private memory area to its child address space in such a way that the start address of this area is specified by the parent and the mapping is not from a file (i.e. page is MAP_ANONYMOUS)???

Hope I have made myself clear.....

will this help you?
Shell :

# man 2 clone

I think clone cannot be used in this particular scenario because of the following reasons:

1)Clone is mainly used to share execution context between the parent and the child. But in our case, the address space of the child has to be private, it need not (and mostly should not) be shared with the parent.

2) A child created with clone(int (*fn) void*,...) will execute the function fn and then terminates. But in our case, the child has to exec to a particular program and continue execution even if the parent dies.