LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 08-20-2009, 08:41 AM   #1
sd136742
LQ Newbie
 
Registered: Aug 2009
Posts: 2

Rep: Reputation: 0
Per-process memory queries on Redhat 5.3 (32 and 64-bit)


Hi There

I have two machines (HP DL380's), with 8Gb of RAM each, and Intel Xeon CPU's. One is installed with Redhat 5.3 32-bit with PAE, and the other with RedHat 5.3 64-bit.

I'm doing some testing with regards to the per-process memory limits. I'm running a script which just executes a never-ending loop increasing the size of a variable with each iteration. Script code is...

VAR="a"
echo "" > /tmp/output.txt
while [ 1 ]
do
VAR=$VAR$VAR
ps -eo pid,rss,vsize >> /tmp/output.txt
done

I'm monitoring the ps output generated in the output.txt.

On the 32-bit system, the process ends with the following message...

xrealloc: cannot reallocate 1073741952 bytes (0 bytes allocated)

At the time the process crashes, the ps output shows a Resident Set Size of 1574060kb, and a VSIZE of 2101664.

On the 64-bit machine, the same error is displayed...

xrealloc: cannot reallocate 18446744071562068096 bytes (0 bytes allocated)

and the ps output shows a RSS of 3146768 and a VSIZE of 4260432.

Therefore, the 32-bit machine can only address 1.5Gb per process of Physical RAM before the process exits. I was under the impression that this should be greater?

The 64-bit machine can only address 3Gb of physical RAM per process. I was under the impression that this should be MUCH higher?

Furethermore, whilst this script was running, I was periodically running 'free -om', which always showed that swap wasn't being used at all. I would of expected swap to be filling up with this script running.

Can anyone explain what's going on here, or am I completly on the wrong track?

Thanks in advance.

Stephen
 
Old 08-20-2009, 10:12 AM   #2
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,125

Rep: Reputation: 1119Reputation: 1119Reputation: 1119Reputation: 1119Reputation: 1119Reputation: 1119Reputation: 1119Reputation: 1119Reputation: 1119
Quote:
Originally Posted by sd136742 View Post
xrealloc: cannot reallocate 1073741952 bytes (0 bytes allocated)

At the time the process crashes, the ps output shows a Resident Set Size of 1574060kb, and a VSIZE of 2101664.
Notice it is a reallocate. So there isn't a large enough contiguous chunk of free virtual address space. There may be smaller chunks of free virtual address space adding up to nearly a GB.

The VSIZE is 2GB. A 32 bit Linux kernel can be configured to limit processes to 2GB virtual, but I don't think your kernel is configured that way. I think you could use 3GB virtual.

Quote:
On the 64-bit machine, the same error is displayed...

xrealloc: cannot reallocate 18446744071562068096 bytes (0 bytes allocated)
Looks like a bug in the code reporting the error. It might not have actually tried to reallocate that much. It might have tried and failed a reasonable amount and reported it wrong.

But maybe it is a bug in the code that actually tried to do the reallocate.

Either way, it appears someone wrote some code for 32 bit and someone recompiled it for 64 bit without testing for places in the code that assumed 32 bit.

Quote:
Therefore, the 32-bit machine can only address 1.5Gb per process of Physical RAM before the process exits.
The limit is on virtual address space, not physical size. Then the physical size is smaller than the virtual address space because of a number of ordinary effects.

So:
1) You have a limit of 3GB of virtual address space.
2) That filled up with just 2GB of virtual memory because the program asked for more in too large a virtual chunk.
3) That occupied only 1.5GB of actual memory for various reasons.

Quote:
The 64-bit machine can only address 3Gb of physical RAM per process.
The limit (or bug) must be in the program you were running (I assume that was bash). Under the conditions you described, the kernel would not limit a process anywhere near that low.

Last edited by johnsfine; 08-20-2009 at 10:15 AM.
 
Old 08-20-2009, 10:31 PM   #3
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,311

Rep: Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040
You need to read this http://kbase.redhat.com/faq/docs/DOC-6571 and possibly follow some of the links as well.
 
  


Reply

Tags
bit, limit, ram, redhat, rss


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I get the "Per process memory limitation" for 64-bit system? kauuttt Linux - Kernel 4 02-26-2009 09:50 AM
Which OS/DB system combo is best to process 8k+ queries per second? (Schoolproject) NL-Stitch Linux - Server 1 11-16-2008 08:45 PM
Intel64 RHEL system max 32 bit process user memory haimy Red Hat 2 06-03-2007 02:26 AM
Which process is making DNS queries? TruckStuff Linux - Networking 4 11-29-2005 05:05 PM
Help Configuring the Memory Used by a Process in RedHat? (Cache Memory on CPU) geogecko Linux - General 3 02-23-2005 04:32 PM


All times are GMT -5. The time now is 08:52 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration