LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-28-2011, 02:22 PM   #1
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Rep: Reputation: 0
Detection Ram to Partition Swap Space in Kickstart installation


Hello,

I'm having some trouble with a kickstart installation script. I am trying to detect the amount of ram in the system but am getting somewhat unpredictable results.

The part of my script I'm using to get amount of RAM in the system is:

RAM=`cat /proc/meminfo | grep MemTotal: | sed 's/^.*MemTotal: \(.*\)kB*$/\1/'`
RAM=`expr $RAM / 1024`

This seems to give me the correct amount of RAM on any virtual machine, but it gives me seemingly random numbers when used on some hardware servers, on other hardware servers it works fine. When i say random, I don't mean slightly smaller or larger than expected, but off by large (gb) amounts.

Any suggestions on how to accomplish accurate RAM detection during a kickstart installation or reasons why my results would be unpredictable with the script I'm using?

Thanks in advance for any help.
 
Old 11-28-2011, 03:05 PM   #2
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Code:
 awk '/MemTotal/{print $2/1024}' /proc/meminfo
But I'd like to see the "off results", together w/ a bare
output of meminfo.



Cheers,
Tink
 
Old 11-28-2011, 04:40 PM   #3
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Results vs Rawinfo

Alright, the output of that portion of my script is RAM=2905 Mb.

The results from a raw cat /proc/meminfo show MemTotal as 4015228 kb. If you need the rest of the meminfo output i can provide it but I figured why clutter up more space unless you need it.

The results from the raw cat /proc/meminfo are correct (there are 4 1gig sticks of ram in this machine) so the problem must be somewhere in my conversion to Mb?

I'll be the first to admit that that portion of my script is cobbled together from tidbits on the net, but i still don't understand why it works on VMs and some hardware servers.

I'm going to try the code you provided tomorrow. Thank you for the help.
 
Old 11-29-2011, 11:29 AM   #4
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Same issue

I tried the code provided and it also works perfectly on VMs, but on the machine i tested (same machine as in the above post) i get 2914.13 Mb of ram which is off by an entire Gb. This is driving me nuts. Any more ideas or help would be appreciated.
 
Old 11-29-2011, 12:10 PM   #5
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Something that may be affecting my outcome that i didn't think about is that this is in the %pre section of the kickstart. Maybe in the pre section ram detection is handled differently?
 
Old 11-29-2011, 12:29 PM   #6
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by vistal View Post
Something that may be affecting my outcome that i didn't think about is that this is in the %pre section of the kickstart. Maybe in the pre section ram detection is handled differently?
Are you running an installer in a non PAE 32bit kernel and then having it install either a PAE kernel or a 64bit kernel?

The numbers you posted suggest you ran your script under one kernel, but got the info from /proc/meminfo while running under a different kernel.

Last edited by johnsfine; 11-29-2011 at 12:32 PM.
 
Old 11-29-2011, 01:11 PM   #7
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
johnsfine,

I suspect you are on the right track. I'm using a CentOS 5.6 i386 install disc. (modify iso/reburn after each revision of my ks.cfg)

I'm not sure what it uses as an install kernel although I suspect it would be the same kernel as the OS? (off to do some research)

The numbers i ran in the previous post were generated by this process:

1) Insert install disc with my ks.cfg

2) Boot from disc

3) My script outputs the amount of RAM and asks (eventually want to do it automatically) how much swap space to partition. (this is where i'm getting the output from my script)

4) I let the install finish (because it had windows on it previously.)

5) Finally, after the install i ran the cat /proc/meminfo from the terminal.
 
Old 11-29-2011, 01:33 PM   #8
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Just booted off of a puppylinux live cd and ran the cat /proc/meminfo which resulted in 2984172kb ram detected. When that is divided by 1024 it is exactly the same number that my script outputs when using Tinksters code: 2914 Mb.

Not entirely sure how this fits in but either
1) the livecd and the installer use a different kernel than the HD installed OS or
2) RAM is handled/detected differently when booted off of a live or run during an install.
 
Old 11-29-2011, 01:58 PM   #9
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by vistal View Post
either
1) the livecd and the installer use a different kernel than the HD installed OS
I'm pretty sure that is the case. I think you could figure out how to change which kernel the livecd and installer use.
Quote:
or
2) RAM is handled/detected differently when booted off of a live or run during an install.
No.

I have a number of comments on various aspects of what you're trying to do:

Very very old 32 bit CPUs can't run a PAE kernel. I doubt you care whether your Linux installer runs on CPUs that old. But if you do care, then you wouldn't want the install time kernel to be PAE. I expect your install image either has multiple kernels or can configure kernels on the fly. Parts of the process you haven't looked into yet probably select a kernel based on hardware info.

I think the whole idea of computing swap partition size based on ram size is flawed. With modern single user Linux systems there is no correlation between ram size and the need for swap space.

If you do want accurate ram size despite running on a different kernel, I suggest parsing the output of
Code:
dmesg | grep e820 | grep usable
That isn't as easy to parse as what you have but it gives the max amount usable by any kernel rather than the amount usable by the current kernel.
Example output from that command:
Code:
 BIOS-e820: 0000000000010000 - 00000000000a0000 (usable)
 BIOS-e820: 0000000000100000 - 00000000cf379000 (usable)
 BIOS-e820: 0000000100000000 - 0000000830000000 (usable)
To get the final answer you need to add up the sizes of those ranges then convert to decimal
0xa0000 - 0x10000 + 0xcf379000 - 0x100000 + 0x830000000 - 0x100000000

Any line where the start of range is 100000000 or greater represents memory than cannot be used by a non PAE 32 bit kernel. If you specifically know the installed kernel will be 32 bit non PAE, you could leave out those lines.

The amount you will see in meminfo will be smaller than what you get this way, by a small percentage plus a small additional amount. I don't know whether you care about that adjustment.

Last edited by johnsfine; 11-29-2011 at 02:21 PM.
 
Old 11-29-2011, 04:27 PM   #10
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
johnsfine,
Thank you very much. I don't think my problem could be explained/answered any more clearly than that. Going to take me some research time/work to digest everything there, but it was extremely helpful. Will update the thread if I have further problems/success.
 
Old 11-29-2011, 04:34 PM   #11
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Okay, been googling a bit and i'm not sure I understand what those ranges are/how to compute/convert them.

Can you suggest any search terms/conversion tools/tutorials that would help me to understand?

Out of curiosity, what do you suggest as an alternative method of calculating swap space? I doubt I could use it because my company "does it this way." Why? I don't know, but convincing a lot of people more senior than me to change methods is probably not an option.
 
Old 11-29-2011, 05:48 PM   #12
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Further clarification on request for information:

I understand that e820 is where the memory map is stored and greping for usable gives us total memory-memory reserved for BIOS=usable memory.

I think that these ranges represent physical memory addresses? How to add, subtract, or convert these ranges is beyond me at the moment. I will keep doing research trying to understand this.
 
Old 11-30-2011, 07:42 AM   #13
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
I don't know how to work with hex numbers in a shell script or awk or other common tools. So I can't tell you how to do that computation in your script.

I already explained how to do it with a calculator if you understand that the "0x" prefix means the following number is hex.

Switch a calculator to hex mode. Key in that sequence of subtracts and adds. Press = or whatever is used in that calculator to get the full result. Switch the calculator back to decimal mode causing it to redisplay the answer (which is mem size in bytes) in decimal.

You got a post in this thread earlier from Tinkster. Maybe he is still reading this thread. I expect he knows how to use common tools such as awk to pull those hex numbers from that grep output and perform that simple computation. Otherwise, you might start another thread asking how to work with hex numbers in a text stream using shell or awk or other common tools.
 
Old 11-30-2011, 10:15 AM   #14
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Untested (my laptop doesn't produce those e820 lines):
Code:
dmesg | awk '/BIOS-e820/&&/usable/{"echo \"scale=10;ibase=F;"toupper($4)"-"toupper($2)"\"|bc -l"|getline diff; total+=diff}END{print total}'

Cheers,
Tink
 
Old 11-30-2011, 11:43 AM   #15
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Didn't work for me
Code:
> dmesg | awk '/BIOS-e820/&&/usable/{"echo \"scale=10;ibase=F;"toupper($4)"-"toupper($2)"\"|bc -l"|getline diff; total+=diff}END{print total}'
sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file
Often I can look at a nearly correct small program in a language I don't know (such as awk) in figure out the correction. But not this time.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Why size of swap space is 2x of RAM??? Why not 1.5x of RAM or 2.5x of RAM??? Saurav Ghosh Linux - Newbie 7 11-01-2011 03:49 AM
Possibility to create swap partition if extended partition is out of space? bhavisavla Linux - Newbie 1 09-12-2011 06:39 AM
Swap space did not function when physical memory ( RAM ) is full rabbit00 Linux - Newbie 19 07-06-2009 09:18 PM
LXer: Linux: Should You Use Twice the Amount of Ram as Swap Space? LXer Syndicated Linux News 1 11-19-2008 05:02 PM
SWAP space=2*RAM p_test Linux - Enterprise 3 04-27-2006 02:22 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 03:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration