Hi

Is it possible to relocate executable(ELF?) to any other address other than 0x8048000?
If yes, how?

Thanks in advanced.
Regards,
Kailas

If executable itself doesn't compiled in PIC (position independent code) relocation is impossible.
If it's compiled (and linked) as PIE (position independent executable) kernel will select where load executable (like it's do it for shared libraries)

Could you explain what you try to achieve ?

I am looking for Address Space multiplexing on linux kernels (specially on 64 bit). On 64 bit kernel, we have large(47 bits) address space available for user programs. However, very few applications really need this.
Thus, if we can keep multiple processes with smaller address space(40 bits, for example) in same page directory, we can avoid CR3 reloading on process switch. This can give many performance benefits. One of them is improved TLB utilization as TLB is flushed on Intel processors due to lack of ASID based TLB support.

Do you think it is possible to achieve this by relocating linux executables using Segmentation?

Thanks & Regards,
Kailas

Frankly now it's first time that I heard about CR3, TLB and ASID, so I have no idea what performance benefits could you gain. But assuming you really need smaller virtual address spaces. Why you need relocate executable - it's hard and executable usually relative small and only one, while there are shared libraries loaded (on modern distro) to the random addresses and there are a lot of them (in case on Gnome/KDE or even just X program) and at least libc and ld-linux in most trivial case

Wouldn't that destroy protection between processes?

If a process happened to try to read or write at an address that belonged to the previous process, and the TLB entry was still present, the CPU would have no way of knowing that read or write was invalid. So it would allow that access to the other process's memory.

Problem here is I don't want just one small address space. I need to keep multiple such address spaces in memory at the same time. Thus, I have to load different address spaces at different offset. I think it can be easily done if we can use segment registers effectively. Currently, segment register for data segment always points to base 0x0(or some standard address). If we set this value different for each the process i.e. 0x for process 1, 1T for process 2 and so on, then we can keep 128 processes in same page directory.

@johnsfine : We can use same segment registers to provide isolation. We can set the limit value for each segment as 1TB. This will automatically raise segment fault when process will try to access others address space.

Do you mean for X86 or just for X86_64?

For X86, I think your whole idea is pointless because the virtual address space isn't big enough. If your idea weren't pointless, I think using segment registers for that would be massively impractical. I can't say for sure you couldn't kludge Linux to make such use of segment registers, but I'm pretty sure.

For X86_64, that use of segment registers is flat out impossible. They just don't work that way. Try reading some documentation from AMD on the X86_64 architecture.

In your earlier post, I guess I accidentally ignored the line
The answer to that is:
X86: Probably no.
X86_64: Absolutely no.

I agree. Thats why I am focussing on x86_64 for this.

I had read that AMD had removed support for segmentation in its new processors. But I believe it is there on Intel processors. Please correct me if this is wrong.

I had read that Intel's version of the X86_64 architecture was a very close copy of AMD's version, and I haven't found any Intel architecture documentation that covers the details I found explained in AMD documentation. I'm sure that Intel documentation exists, I just didn't try hard enough to find it, because I didn't expect any difference. I'll try again later.

Maybe you're right about Intel X86_64 (though I still doubt it). If so, it would still be VERY difficult to change X86_64 Linux to use segmentation.

Linux already does this using virtual memory addressing and memory paging, etc. Just because every program starts at the same address doesn't mean it actually occupies the same physical address in memory.

As shown above, this is totally irrelevant in a multi-tasking operating system.

I think you're worrying about a problem that doesn't exist.

I read about this in Intel manual. Unfortunately, segment registers behave differently on all 64 bit processors. They are all set to base zero and hence can not be used as I assumed.

Can you suggest some other way to achieve this?

Are you trying to write your own operating system, or are you just assuming that Torvalds doesn't understand multitasking issues in large memory spaces?

To clarify that point for anyone else trying to follow this thread, you mean that Intel x86_64 segment registers behave the same as AMD x86_64 segment registers and both are very different from segment registers in x86 32-bit architecture.

I strongly doubt that there is any way.

I have only the vaguest understanding of what "ASID based TLB support" is. I understand exactly why it is needed and what it accomplishes, but I have no idea how it is implemented nor which CPU models implement it nor what kernel support exists.

On a CPU model without it, I don't think there is any work around to avoid TLB flushes when changing between processes (without violating protection).

The problem is real. The fact that Torvalds has not chosen to deal with some problem does not, by itself, mean nothing can or should be done by someone else.

In this particular case, I personally don't think there is anything that can or should be done. But I'm still bothered by the fact that you seem to be reaching that same "do nothing" answer without bothering to understand the topic being discussed. It distracts the thread and reduces the chance (already slight unfortunately) that someone who understands some detail I might have missed would jump in with some constructive suggestions for the OP.

Kailas,

I'm going to take one final stab at getting a response from you.

First of all, I'll point you to: http://linux-mm.org/ This page is the jumping-off point for intrepid neophyte memory management mavens. There is a link to an explanation to how the internals of Linux memory management works. After reading that, if you are still convinced that you have come across something new, then I'd suggest that you join the linux-mm mailing list (also linked from that page). You might want to look through the list's archives first, though, as I suspect that the developers are just as busy as the ones I encountered on the linux-ide group back when I was helping test the sata_inic162x board. Still, if you are sure...

I'd also like to point out an issue with your thesis of extremely large monolithic programs. Usually when we find ourselves in such a position it is way past the time to rethink the program's architecture. Normally we would want to put as much data as possible into our database management system of choice, and only keep, in memory, what is truly needed. There are also techniques using mmap (or whatever you favor) that would allow you to swap between multiple large mapped data sets. When it comes to the program itself, if it's truly that large, you should benefit from splitting it up into multiple programs and have them communicate with each other using your favorite inter-process communication protocol.

Edited: spelling