[SOLVED] C two dimensional array, contiguous memory or not ?
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
C two dimensional array, contiguous memory or not ?
Is there any big advantage to defining a C two
dimensional array e.g. array[rows][cols] so
that it is to use contiguous memory ?
Wouldn't non-contiguous memory be easier for
the system to allocate ?
A small array, like array[50][100], might not matter,
but what about when you need a seriously large array,
like maybe array[1000][7000] or bigger ?
I assume I'd need a loop to free this memory, row by
row, once I've done with it, or would just one call
like this, free( array ); be good enough ?
Is there any big advantage to defining a C two
dimensional array e.g. array[rows][cols] so
that it is to use contiguous memory ?
One thing comes to mind: simpler, more flexible code.
With single contiguous memory, you can reference your array with subscripts and/or uni-dimensional arithmetic (e.g. "array[row# * rowSize + column#]" ). With non-contiguous, you would have to reference the memory like so "(array[row#])[column#]" -- to force the compiler to resolve the array[row#] as a pointer to individual elements, and then apply the subscript for the column.
Quote:
Wouldn't non-contiguous memory be easier for
the system to allocate ?
Only if the system memory is fragmented--which would probably only occur if a large percentage of the system memory is in use. And on top of that, only if the system then chooses not to move memory to swap space/virtual memory in an attempt to satisfy the active process's memory demands.
Quote:
I assume I'd need a loop to free this memory, row by
row, once I've done with it, or would just one call
like this, free( array ); be good enough ?
Yes, you would need a loop to free it. For every memory reservation call, you must have a corresponding free() to avoid a memory leak.
Try the following code once you are certain you understand what it's doing:
Code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main( int argc, char *argv[] )
{
char **doubleArray;
int arrayIndex;
FILE *meminfo;
char *lineBuffer;
size_t lineBufferSize;
long int freeMemory;
long int freeMemoryStart;
int loopCounter;
freeMemoryStart = 0;
loopCounter = 0;
do
{
/* reset our free memory variable */
freeMemory = 0;
/* open /proc/meminfo to read system memory information.
available system memory is reported on a line that starts with
"MemFree:" -- read lines from the file until we find that particular
line. We don't care about the units--just the numeric value, which
drop if there is a memory leak */
meminfo = fopen( "/proc/meminfo", "r" );
do
{
lineBuffer = NULL;
getline( &lineBuffer, &lineBufferSize, meminfo );
if( strstr( lineBuffer, "MemFree:" ) )
freeMemory = strtol( lineBuffer + strlen( "MemFree:" ), NULL, 10 );
free( lineBuffer );
} while( freeMemory == 0 );
fclose( meminfo );
/* store a baseline free memory so we don't bring the computer to a
grinding halt if a memory leak is present */
if( freeMemoryStart == 0 )
freeMemoryStart = freeMemory;
/* Display the free memory info */
fprintf( stdout, "[%5d] Current free memory: %ld\n", loopCounter, freeMemory );
/* Do the allocations: one allocation to store pointers to the "rows"
and then another allocation for each row.
Reserve sufficient memory that we don't need millions of allocations
to see a drop (if a leak is present), but not so much that one
one loop is all the system can afford to handle.
In short: a 1024x1024 array (or 1 MiB) */
doubleArray = (char **)malloc( sizeof( char * ) * 1024 );
for( arrayIndex = 0; arrayIndex < 1024; arrayIndex++ )
doubleArray[arrayIndex] = (char *)malloc( sizeof( char ) * 1024 );
/* Only free the "containing" array */
free( doubleArray );
loopCounter++;
/* Break the loop if the system memory drops by 10% of the original
starting memory (or more). Or, if no leak, break after 10,000 loops */
} while( ( freeMemoryStart - freeMemory < freeMemoryStart / 10 ) &&
( loopCounter < 10000 ) );
if( loopCounter == 10000 )
fprintf( stdout, "No memory leak!\n" );
else
fprintf( stdout, "Lost memory in loop!\n" );
return 0;
}
If you want it to be flexible, create your own type, like:
Code:
typedef stuct MyTable {
int n;
int m;
int **v;
} MyTable
int *AccessTable (MyTable *m, int i, int j);
MyTable *AllocTable (int n, int m);
void FreeTable (MyTable *t);
Wouldn't non-contiguous memory be easier for
the system to allocate ?
If "system" means the OS, absolutely not. Each process has its own virtual address space. There is always a layer of page mapping between the process virtual address space and the physical address space. So contiguous in process virtual does imply contiguous in physical.
In 32 bit, if you allocate and deallocate enough memory within one process that the process virtual address space is fragmented, or you want an array so large (near 3GB) that initial fragmentation from .so loading matters, then a non contiguous array might fit where contiguous doesn't. But if you are using memory that heavily, why would you use 32 bit rather than 64 bit.
Quote:
A small array, like array[50][100], might not matter,
but what about when you need a seriously large array,
like maybe array[1000][7000] or bigger ?
Unless each element of the array is pretty big, 1000x7000 is still small and you wouldn't worry about even 32 bit fragmented virtual memory. 1000x7000 doubles (64 bit floating point) is 56MB. You really shouldn't be worrying about virtual address space fragmentation unless you have contiguous objects of hundreds of MB. In 64 bit, you don't worry about virtual address space fragmentation at all.
Quote:
Originally Posted by Dark_Helmet
Only if the system memory is fragmented--which would probably only occur if a large percentage of the system memory is in use. And on top of that, only if the system then chooses not to move memory to swap space/virtual memory in an attempt to satisfy the active process's memory demands.
Because of paging, system memory "fragmented" is a meaningless or at least irrelevant concept.
Process memory is always "virtual". The system does not use "virtual" memory only as a fall back when physical memory is full. "virtual" memory is always used and always makes physical contiguity of memory irrelevant for virtual contiguity.
So far as I understand, Linux does not dynamically mix/convert 4KB pages with 2MB pages. When you allocate a large contiguous data structure on a system with large physical ram, there would be better performance if 2MB pages could be used instead of 4KB pages. If Linux managed 4KB vs. 2MB pages dynamically, then physical contiguity would matter because 512 4KB pages could be used as 1 2MB page only if they are contiguous and aligned. I think Linux would be a better OS if it did dynamically manage 4KB/2MB paging. But I haven't read about any efforts toward that. Linux has some crude support for 2MB pages that is less useful than dynamic support would be.
Use "C"plus-plus, which gives you a rich abundance of "container classes" to choose from.
Another way to think of a "two-dimensional array" is that it is a storage class which takes a vector of integers as its input. The traditional implementation allocates a fixed size chunk of memory and thus imposes hard limits upon the range of each vector while simultaneously occupying storage for every element even if not every element is used.
Various storage classes exist which relax these limitations in different ways. You don't have to write any of them...
Because of paging, system memory "fragmented" is a meaningless or at least irrelevant concept.
I will defer to your greater familiarity with the subject. My knowledge of virtual memory is derived from my college coursework some 10 years ago. That topic (along with disk fragmentation) was touched on only briefly for my degree. As a result, only the most basic approaches were discussed.
C two dimensional array, contiguous memory or not ?
Thank you all for you great replies ! Lots of new and interesting ideas.
You guys are really smart people. It looks like contiguous memory arrays
would be best and I'll use loops to free each array pointer. I'm using
pointer-to-pointer for the 2 dimensional array.
I'd say this issue has been solved. Thanks !
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.