LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 05-16-2016, 02:59 AM   #1
Borneq
Member
 
Registered: Aug 2014
Posts: 36

Rep: Reputation: Disabled
Is possible to alloc large amount of memory in ONE block?


especially many gigabytes in 64-bit Linux?
For example, system has 6 GB free memory. Is possible to alloc it in ONE block or 60 100-mega blocks ?
Many block very complicate algorithms, for example : one heap structure: parent index = 1/2 child index, child index = 2*parent index.
I want have one large heap, but if will many blocks, algorithm will difficult and inefficient.
 
Old 05-16-2016, 04:06 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,842

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
what language? have you tried it at all? How?
 
Old 05-16-2016, 04:29 AM   #3
Borneq
Member
 
Registered: Aug 2014
Posts: 36

Original Poster
Rep: Reputation: Disabled
Language c++
 
Old 05-16-2016, 08:29 AM   #4
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,842

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
have you tried it at all? How?
 
Old 05-16-2016, 12:33 PM   #5
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941
I would frankly suggest that you probably should not pursue this approach. Here are a handful of possible reasons why:

It's actually virtual storage. Your application's entire memory-space is carved up into "pages" which are demand-loaded from external storage. When you use up the total number of physically resident pages that you can presently have, pages begin to get "stolen." Each page-operation requires milliseconds to complete ... apiece.

Locality of reference: This is the pivotal assumption upon which the entire concept of virtual memory is built, and it's the key to building production systems that consistently perform well. The virtual memory manager assumes that "the next memory reference" made by your application "probably will be nearby to" some other memory reference that the same app made "recently."

If you need "arrays" in your source-code, C++ allows you to use that syntax ... even if the underlying implementation is different.

It is generally unwise to "roll your own." C++ has a truly vast standard library of classes, with plenty more available from well-known Internet sources.
 
Old 05-18-2016, 01:30 PM   #6
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Quote:
Originally Posted by sundialsvcs View Post
I would frankly suggest that you probably should not pursue this approach. Here are a handful of possible reasons why:

It's actually virtual storage. Your application's entire memory-space is carved up into "pages" which are demand-loaded from external storage. When you use up the total number of physically resident pages that you can presently have, pages begin to get "stolen." Each page-operation requires milliseconds to complete ... apiece.

Locality of reference: This is the pivotal assumption upon which the entire concept of virtual memory is built, and it's the key to building production systems that consistently perform well. The virtual memory manager assumes that "the next memory reference" made by your application "probably will be nearby to" some other memory reference that the same app made "recently."

If you need "arrays" in your source-code, C++ allows you to use that syntax ... even if the underlying implementation is different.

It is generally unwise to "roll your own." C++ has a truly vast standard library of classes, with plenty more available from well-known Internet sources.
I concur with these thoughts. If you are using a general purpose OS environment, then don't try to use the maximum available and stress the limits. If you are working on a custom system, an embedded one, where memory management is performed, you can pre-define a memory segment to be reserved when you construct the system. That is how things such as this are performed. A custom system is constructed where this large memory segment is reserved solely for the use of your custom program.

A general purpose OS just isn't going to provide you with maximum services to protect things and also allow the location of the memory to be consistent. I know you weren't asking for that, but in my opinion, this would be the next step when you tried to debug the fruits of your first level effort.
 
Old 05-19-2016, 07:53 AM   #7
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941Reputation: 3941
C++'s standard (and contributed) libraries are fairly stuffed with good "container classes" that will enable applications to have the functional illusion of "an unlimited, n-dimensional storage space within which to work," which some applications do need. But, they do it efficiently, being extremely mindful of the vagaries of virtual storage ... particularly the fundamental(!) concern of locality of reference.

"Don't fret over nanoseconds" here. There is no price-to-be-paid by slightly longer code paths taking a few billionths of a second more. No, the real price is measured in "multiple milliseconds," and they occur with every page fault.

Uncontrolled, or simply unplanned-for, page fault activity is rudely called, "thrashing." Or, as I used to say back in the days when a 20-megabyte disk drive was about the size of an apartment's washing-machine, "MaytagŪ Mode." We literally had one of those devices slowly juggle itself to the very edge of the raised floor, over in the corner where a few tiles were removed, and ... (The spinning disk broke loose from the capstan and very nearly went sailing out across the machine room.)

The performance-degradation curve caused by thrashing is variously called "elbow-shaped" or simply, "hitting the wall." It basically goes straight to Hell straight up, exponentially. Poorly designed applications which need lots of RAM are especially vulnerable to it, the moment they come out of the memory-rich "developer's environment" and an attempt is made to deploy them to production.

Last edited by sundialsvcs; 05-19-2016 at 07:56 AM.
 
Old 05-19-2016, 09:11 AM   #8
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: Fedora
Posts: 4,140

Rep: Reputation: 1263Reputation: 1263Reputation: 1263Reputation: 1263Reputation: 1263Reputation: 1263Reputation: 1263Reputation: 1263Reputation: 1263
You can reserve memory on the kernel command line using memmap=, then create a virtual address in your userspace program that points to it. You will need to rebuild the kernel to allow access via /dev/mem.

http://lwn.net/Articles/267427/
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
max memory alllocation bu ,alloc manohar Linux - Software 6 07-10-2012 11:48 AM
memory wouldn't alloc in C dracuss Programming 4 12-18-2008 12:10 PM
shmat() Failure While Using a Large Amount of Shared Memory jayanth Programming 10 02-25-2008 12:03 PM
A large amount of problems DaBlade Linux - General 4 12-04-2004 05:47 AM
Large drive- too much space lost by large block size? tome Linux - General 5 12-14-2001 01:32 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 12:23 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration