LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 09-21-2017, 12:30 AM   #1
diya26
LQ Newbie
 
Registered: Jul 2015
Posts: 13

Rep: Reputation: Disabled
How to understand the virtual Memory Layout ?


Hi,

I am using custom board with DDR size of 512 MB. I need to reserve a large buffer of physically RAM from the kernel and be able to read /write to that hard-coded physical address.I need to reserve 300-400MB for application.

I have gone through below link to understand the virtual layout,
https://unix.stackexchange.com/quest...in-dmesg-imply

But i am not able to understand on upto how much memory i can access from the user space directly.
Does the "lowmem" mentioned in layout refer to physical memory of 512 MB. If so , will i am be able to use the complete memory ?

I understand that 32 bit machine allocate 4GB space out of which 1GB for kernel space and 3GB for User space access. But i am not able to figure out the difference of this 4GB and my Ram 512 MB. And how this 512 MB ram fit in kernel space .


Below is my virtual Mapping ,

Code:
Memory: 486136K/524288K available (6311K kernel code, 271K rwdata, 1948K rodata, 1024K init, 223K bss, 21768K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:
vector  : 0xffff0000 - 0xffff1000   (   4 kB)
fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
vmalloc : 0xe0800000 - 0xff800000   ( 496 MB)
lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
modules : 0xbf000000 - 0xbfe00000   (  14 MB)
  .text : 0xc0008000 - 0xc0910dc4   (9252 kB)
  .init : 0xc0a00000 - 0xc0b00000   (1024 kB)
  .data : 0xc0b00000 - 0xc0b43e10   ( 272 kB)
   .bss : 0xc0b43e10 - 0xc0b7ba64   ( 224 kB) code here

Can anyone explain me about this layout in detail,
 
Old 09-21-2017, 10:21 AM   #2
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,599
Blog Entries: 4

Rep: Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905
Why is this buffer "hard coded?" Is it being accessed, say, by some hardware device?

Please give us more details about your use-case so that we can better answer your question.
 
Old 09-26-2017, 01:41 AM   #3
diya26
LQ Newbie
 
Registered: Jul 2015
Posts: 13

Original Poster
Rep: Reputation: Disabled
We have 512MB DDR 0n our board. The memory address start from 0 to 0x20000000. We have loaded u-boot and booted linux 4.0 .Through application we need to Read /write into any address of physical memory (512MB).(i.e., I need to write into any physical address e.g.,0x100000 of some 1000Bytes and read back the data from the same address).

I used mmap to map physical address and perform read/write.It work fine with small memory size like 20MB but application was not working stable when i map huge memory size like 256MB and above.Got kernel hang or crash frequently.

I am new to linux environment and i am not able to get enough information about virtual memory layout.Below are my Queries,

1) How kernel virtual memory mapped with physical memory.
2) Upto how much physical memory regions i can access from user space and is it safe to access without crashing or affecting kernel memory.
3) Will i be able to access the physical memory with direct address like (0x1000000 0x8000000 )from user space.
4) How much size i can use through malloc and how to know the map size. Do i need to get this information from virtual layout ? but no .heap memory allocated in virtual layout ?
 
Old 09-26-2017, 02:17 AM   #4
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,629

Rep: Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265
1) virtual memory is dynamically mapped to physical memory (on demand) - as far as I know
2) from user space you are not able to use directly any regions of physical memory, but some memory will be associated to the process and that will be either put into physical ram or swapped out (if possible). This cannot conflict with kernel memory.
3) I don't think so, you need special kernel driver for that - if I know it well
4) malloc will return NULL if no more space can be allocated, The virtually available memory is usually more than RAM+swap.

probably you will find this page useful: www.linuxatemyram.com
 
Old 09-26-2017, 03:32 AM   #5
diya26
LQ Newbie
 
Registered: Jul 2015
Posts: 13

Original Poster
Rep: Reputation: Disabled
Hi,

Thanks for reply,

Can you tell me what lowmem (0xc0000000 - 0xe0000000( 512 MB)) mean in virtual layout ? Is it specifying the physical memory ?

I feel difficult to understand this virtual layout.

Virtual layout seem to be of 4GB for 32bit machine. And the physical memory which i am using is only 512MB.
How kernel uses this physical memory in kernel space for running its process. According to my understanding,kernel should allocate some memory for any process to execute.
So how it allocate the physical memory and how will end user knows it such that application will not touch that memory region or use that region to avoid crash.

Correct me if i am wrong.
 
Old 09-26-2017, 05:59 AM   #6
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,629

Rep: Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265
32bit machine can handle 4GB of memory altogether. This can be (partially) virtual or fully physical.
If you want to know how can the kernel start a new process please read how fork/clone work. https://en.wikipedia.org/wiki/Fork_%28system_call%29, https://www.bottomupcs.com/fork_and_exec.xhtml

http://www.tldp.org/LDP/tlk/mm/memory.html
 
Old 09-26-2017, 08:00 AM   #7
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,599
Blog Entries: 4

Rep: Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905
I think that you need to use some kind of kernel interface – such as, say, a loadable virtual device. Applications can "open" the device, "seek" to any particular point, then "read" or "write" data. Of course the virtual device restricts itself to the confines of the memory buffer, the physical location of which you already know.

The existing /dev/kmem virtual device is almost identical in concept. (But I would not attempt to use that ...)

User processes do not have direct access to physical memory, and although it would be possible to "map" the physical storage into the virtual-storage image of a particular process, I think that the virtual-device interface would be easier.

Last edited by sundialsvcs; 09-26-2017 at 08:03 AM.
 
Old 09-27-2017, 10:04 AM   #8
diya26
LQ Newbie
 
Registered: Jul 2015
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
32bit machine can handle 4GB of memory altogether. This can be (partially) virtual or fully physical.
If you want to know how can the kernel start a new process please read how fork/clone work. https://en.wikipedia.org/wiki/Fork_%28system_call%29, https://www.bottomupcs.com/fork_and_exec.xhtml

http://www.tldp.org/LDP/tlk/mm/memory.html
How 32bit machine can handle 4GB space when physical memory itself is 512MB. Moreover, i am able to allocate only 4MB through kmalloc in kernel space.

I want to know exactly how much physical memory space i will be able to use it from user space.

Can you brief me on this ,
 
Old 09-28-2017, 12:30 AM   #9
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,629

Rep: Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265Reputation: 7265
4 GB is the theoretical maximum, but obviously the real hardware will handle only the available memory (but not more that 4GB).
 
1 members found this post helpful.
Old 09-28-2017, 01:39 AM   #10
diya26
LQ Newbie
 
Registered: Jul 2015
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
4 GB is the theoretical maximum, but obviously the real hardware will handle only the available memory (but not more that 4GB).
You mean to say that 512 MB can be accessed from kernel space as well as from user space. But will not know exact which physical address it is current accessing (like 0x100000 or 0x8000000). And all physical memory direct address will be as accessed as virtual address in both kernel and user space (like mmap).

And from the Memory map shown below , Memory use 486136K out of 524288K available but its missing 37k out of total memory and why its not exactly using the whole memory. And from Kernel memory map, i understand pkmap to vector uses 1GB kernel space. And remaining 3GB for USER space.And I am able to do malloc upto 300MB in user space but in kernel space kmalloc upto 4MB only. Why i am not able to assign larger memory size in kernel space. And kernel crash,if i use vmalloc of 32MB size,

Quote:
Memory: 486136K/524288K available (6311K kernel code, 271K rwdata, 1948K rodata, 1024K init, 223K bss, 21768K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0xe0800000 - 0xff800000 ( 496 MB)
lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
.text : 0xc0008000 - 0xc0910dc4 (9252 kB)
.init : 0xc0a00000 - 0xc0b00000 (1024 kB)
.data : 0xc0b00000 - 0xc0b43e10 ( 272 kB)
.bss : 0xc0b43e10 - 0xc0b7ba64 ( 224 kB) code here
Can you suggest about this,

Correct me if my understanding is wrong.
 
Old 09-29-2017, 12:58 PM   #11
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,599
Blog Entries: 4

Rep: Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905
Actually, no: physical RAM-addresses can't be developed or used by user-land programs.

All user programs run in a virtual-memory environment and cannot escape from it.

As I previously suggested, an obvious and ready solution to this problem is a virtual device, or even a virtual filesystem such as kmem.

There are plenty of already-perfected examples available to you in both cases. Simply 'cabbage' one out of the kernel source-trees, 'whack-a-mole' on it for a little while until it turns into what you need, and use it. Either way, "you do not need to (re-)invent this."
 
1 members found this post helpful.
  


Reply

Tags
linux kernels, memory management


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
linux hexedit layout trying to understand deleted07 Linux - Newbie 3 01-18-2013 06:45 AM
Linux shared memory segment access problem and x86 Virtual Memory layout. regmee Linux - Kernel 1 08-23-2008 12:11 AM
Difference between resident memory,shared memory and virtual memory in system monitor mathimca05 Linux - Newbie 1 11-11-2007 04:05 AM
Layout of Memory KissDaFeetOfSean Programming 3 07-17-2005 10:42 PM
G++ memory layout of the classes shibdas Programming 4 09-20-2004 07:17 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 12:43 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration