LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-30-2011, 01:16 PM   #16
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928

Hi John,

What version of awk, which shell, and which bc?


Hmmm ... copy & paste from my post into a terminal here at work renders this
(physically incorrect, the machine has 4GB) result:
Code:
$ dmesg | awk '/BIOS-e820/&&/usable/{"echo \"scale=10;ibase=F;"toupper($4)"-"toupper($2)"\"|bc -l"|getline diff; total+=diff}END{print total}'
2681484075

Cheers,
Tink

Last edited by Tinkster; 11-30-2011 at 01:20 PM.
 
Old 11-30-2011, 02:02 PM   #17
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 428Reputation: 428Reputation: 428Reputation: 428Reputation: 428
Quote:
Originally Posted by Tinkster View Post
What version of awk, which shell, and which bc?
It is probably a bad idea to rely on dmesg output.
On my laptop I have the following:
Code:
$ dmesg  | awk  '/BIOS-e820/&&/usable/'
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000bb6a1000 (usable)
[    0.000000]  BIOS-e820: 00000000bb6a7000 - 00000000bb7ba000 (usable)
[    0.000000]  BIOS-e820: 00000000bb80f000 - 00000000bb908000 (usable)
[    0.000000]  BIOS-e820: 00000000bbb0f000 - 00000000bbb19000 (usable)
[    0.000000]  BIOS-e820: 00000000bbb1f000 - 00000000bbb5f000 (usable)
[    0.000000]  BIOS-e820: 00000000bbb9f000 - 00000000bbbe2000 (usable)
[    0.000000]  BIOS-e820: 00000000bbbff000 - 00000000bbc00000 (usable)
[    0.000000]  BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
so we should use fields $4 and $6 (not $2, $4). I think `ibase' should be set to 16 (not F=15):
Code:
$ dmesg | awk '/BIOS-e820/&&/usable/{"echo \"scale=10;ibase=16;"toupper($6)"-"toupper($4)"\"|bc -l"|getline diff; total+=diff}END{print total}'
4220355584
The result seems to be correct.
 
Old 11-30-2011, 02:26 PM   #18
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Quote:
Originally Posted by firstfire View Post
I think `ibase' should be set to 16 (not F=15):
Ooops - thanks for catching that ... that's what I get for tinkering ~ 5am :P


:D


Cheers,
Tink
 
Old 11-30-2011, 04:07 PM   #19
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by Tinkster View Post
What version of awk, which shell, and which bc?
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)

I have no getline

Is it a program? Or part of bash?
 
Old 11-30-2011, 04:21 PM   #20
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Quote:
Originally Posted by johnsfine View Post
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)

I have no getline

Is it a program? Or part of bash?
An awk built-in function. Something must be off with the quotes being copied
into your environment.



Cheers,
Tink
 
Old 12-01-2011, 09:49 AM   #21
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
First,
Thank you guys for all of your help. As a mediocre practitioner of bash scripting I'm in a bit over my head, but everything else in my install script works so i'm determined to make this part work too.

Using the code from firstfire doesn't work on my vms, which i'm looking into now (gives me the EOF error johnsfine was getting).

It does give me an output on my physical machine when run from a live cd but its a little bit puzzling:
-3086333952

Any ideas why it would return a negative output?
 
Old 12-01-2011, 10:27 AM   #22
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Quote:
Originally Posted by vistal View Post
First,
Thank you guys for all of your help. As a mediocre practitioner of bash scripting I'm in a bit over my head, but everything else in my install script works so i'm determined to make this part work too.

Using the code from firstfire doesn't work on my vms, which i'm looking into now (gives me the EOF error johnsfine was getting).

It does give me an output on my physical machine when run from a live cd but its a little bit puzzling:
-3086333952

Any ideas why it would return a negative output?

Not w/o seeing the input data; can only guess that your dmesg output
varies from John's.


Cheers,
Tink
 
Old 12-01-2011, 10:48 AM   #23
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Using spaces as delimiters and trying to know the field number from start of line seems to be far too optimistic about dmesg output format consistency.

Since I don't know awk at all, I can't make a constructive suggestion.

If I were parsing this in C++ I would look for the two long groups of valid hex digits that come after the e820. I think the first two such groups should safely be the correct two. But I would probably include a bit more sanity check than that.
 
Old 12-01-2011, 11:52 AM   #24
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 428Reputation: 428Reputation: 428Reputation: 428Reputation: 428
Hi.
Maybe this will work
Code:
$ dmesg | awk  '/BIOS-e820/&&/usable/{gsub(/([^:]+: )|\(usable\)/, ""); "echo \"scale=10;ibase=16;" toupper($0)"\"|bc -l"|getline diff; total-=diff}END{print total}'
4220355584
 
Old 12-01-2011, 02:13 PM   #25
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Quote:
Originally Posted by johnsfine View Post
Using spaces as delimiters and trying to know the field number from start of line seems to be far too optimistic about dmesg output format consistency.

Since I don't know awk at all, I can't make a constructive suggestion.

If I were parsing this in C++ I would look for the two long groups of valid hex digits that come after the e820. I think the first two such groups should safely be the correct two. But I would probably include a bit more sanity check than that.
Taken that input on ... this might work (as long as dmesg' output
is half-way sane):
Code:
/BIOS-e820/&&/usable/{
  for(i=1;i<=NF;i++){
    if($i ~ /[0-9a-fA-F]{16}/){
      "echo \"scale=10;ibase=16;"toupper(gensub(/0?[xX]?([0-9a-fA-F]{16})/,"\\1","1",$i))"\"|bc -l"|getline tmp
      if(a[NR]){
        a[NR]+=tmp
        total+=a[NR]
      }else{
        a[NR]-=tmp
      }
    }
  }
}
END{
  print total
}
Save as whatever.awk, and run it like this:
Code:
dmesg | awk --re-interval -f whatever.awk 
4183199744


Cheers,
Tink
 
Old 12-04-2011, 02:33 PM   #26
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Any news on this front, OP?


Cheers,
Tink
 
Old 12-05-2011, 12:21 PM   #27
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Will update soon. Testing the output in different scenarios/trying to understand whats going on and still having some problems. As soon as I rule out any stupid behavior on my part I will update you guys with the information I have available.
 
Old 12-05-2011, 06:34 PM   #28
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Alright, I am still having some issues and will provide all of the info I can.

First, when included in my script i get 0.

This is where i echo out the awk file:
Code:
echo "/BIOS-e820/&&/usable/{" > ram.awk
echo '   for(i=1;i<=NF;i++){' >> ram.awk
echo '      if($i ~ /[0-9a-fA-F]{16}/){' >> ram.awk
echo '        "echo \"scale=10;ibase=16;"toupper(gensub(/0?[xX]?([0-9a-fA-F]{16})/,"\\1","1",$i))"\"|bc -l"|getline tmp' >> ram.awk
echo '        if(a[NR]){' >> ram.awk
echo '            a[NR]+=tmp' >> ram.awk
echo '            total+=a[NR]' >> ram.awk
echo '          }else{' >> ram.awk
echo '            a[NR]-=tmp' >> ram.awk
echo '          }' >> ram.awk
echo '        }' >> ram.awk
echo '      }' >> ram.awk
echo '     }' >> ram.awk
echo 'END{' >> ram.awk
echo '  print total' >> ram.awk
echo '}' >> ram.awk
This is where i call the awk file:

Code:
RAM=dmesg | awk --re-interval -f ram.awk
And finally this is where I print the RAM variable (there is no other manipulation of the RAM variable anywhere else in the script:

Code:
echo "Total space on drive (in Mb): $SPACE"
echo "Remaining space available (in Mb): $SPACE3"
echo "System RAM (in Mb): $RAM"
echo -n "Space allocated for swap (in Mb):"
read VAR3
and the output is 0 . Strangely, when first testing i had accidentally included an = in the call (RAM=dmesg | awk --re=interval -f ram.awk) and it provided me with 2914.13. Not sure if that means anything or not though.

Testing on the puppy linux live cd:

Contents of ram.awk: (srry bout the formatting, it didnt like the copy paste into the shell, but other than the tabs being off it didn't look like it changed anything)
Code:
/BIOS-e820/&&/usable/{
  for(i=1;i<=NF;i++){
      if($i ~ /[0-9a-fA-F]{16}/){
            "echo \"scale=10;ibase=16;"toupper(gensub(/0?[xX]?([0-9a-fA-F]{16})/
                  if(a[NR]){                                                    
                          a[NR]+=tmp                                            
                                  total+=a[NR]                                  
                                        }else{                                  
                                                a[NR]-=tmp                      
                                                      }                         
                                                          }                     
                                                            }                   
                                                            }                   
                                                            END{                
                                                              print total       
                                                              }
The output:
sh-4.1# dmesg | awk --re-interval -f ram.awk
3084648448

Which is not 4 gig .

Here is the output of dmesg from the puppy linux live cd:
Code:
sh-4.1# dmesg | grep e820
 BIOS-e820: 0000000000000000 - 000000000009b800 (usable)
 BIOS-e820: 000000000009b800 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000b7ec0000 (usable)
 BIOS-e820: 00000000b7ec0000 - 00000000b7ee0000 (reserved)
 BIOS-e820: 00000000b7ee0000 - 00000000b7ee3000 (ACPI NVS)
 BIOS-e820: 00000000b7ee3000 - 00000000b7ef0000 (ACPI data)
 BIOS-e820: 00000000b7ef0000 - 00000000b7f00000 (reserved)
 BIOS-e820: 00000000b8000000 - 00000000c0000000 (reserved)
 BIOS-e820: 00000000f0000000 - 00000000f4000000 (reserved)
 BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
ACPI: HPET id: 0x10de8201 base: 0xfeff0000
sh-4.1#
Finally, i tested this on the already installed cent os (referred to previously in the post as HD installed)
contents of ram.awk:
Code:
/BIOS-e820/&&/usable/{
  for(i=1;i<=NF;i++){

    if($i ~ /[0-9a-fA-F]{16}/){
      "echo \"scale=10;ibase=16;"toupper(gensub(/0?[xX]?([0-9a-fA-F]{16})/,"\\1","1",$i))"\"|bc -l"|getline tmp
      if(a[NR]){
        a[NR]+=tmp

        total+=a[NR]
      }else{
        a[NR]-=tmp
      }
    }
  }
}
END{
  print total
}
[root@localhost ~]# dmesg | awk --re-interval -f ram.awk
sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file

So I checked the version of both bash and awk:

GNU bash, version 3.2.25(1)-release (i686-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.

GNU Awk 3.1.5
Copyright (C) 1989, 1991-2005 Free Software Foundation.


My thoughts: Kinda lost right now as my understanding of whats going on in that awk script = 0. It is quite possible I have screwed something up in the script by copy/paste or echoing it out. It is also possible that I am not echoing it out early enough in the script (kickstart is a picky thing), but i am doing it before i call it in the %pre section. I would be more inclined to experiment along those lines if the output from the live cd was correct. Other than that, because i don't know how the script is working, I feel quite lost. I think my next trouble shooting attempt is going to be to call dmesg | grep e820 in my script so i can look at the output of that from the installation kernel, but that will have to wait until tomorrow.

Strange that this is the part of my script I am hung on as I thought this part would be much easier than the rest, but everything else is already done Any further assistance or suggestions anyone can provide would be greatly appreciated.

Thanks

Last edited by vistal; 12-05-2011 at 06:40 PM.
 
Old 12-05-2011, 06:38 PM   #29
vistal
LQ Newbie
 
Registered: May 2011
Posts: 25

Original Poster
Rep: Reputation: 0
Just realized that there is something massively wrong with the awk file in the puppy linux test. First thing I will try tomorrow is confirm if that was just something in the copy/paste/e-mail to myself/post process, or if I was actually running the command with that as the awk file.

Last edited by vistal; 12-05-2011 at 06:43 PM.
 
Old 12-06-2011, 07:42 AM   #30
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by vistal View Post
Code:
RAM=dmesg | awk --re-interval -f ram.awk
That does not look right to me (doesn't look like RAM could capture the final result), but I should let some bash expert answer. I really don't know that stuff.

Quote:
sh-4.1# dmesg | awk --re-interval -f ram.awk
3084648448

Which is not 4 gig .
The script appears to be giving you very close to correct info. You don't have near 4GB usable.

You probably have 4GB physically installed, so dmidecode would report 4GB, and likely the BIOS reported 4GB. But the BIOS didn't enable 4GB for the OS to use.

Hopefully, a BIOS menu setting ("memory remapping" or something like that) could be changed to enable more ram. But maybe the motherboard doesn't support more than 3 and a fraction GB of actual ram.

Quote:
Here is the output of dmesg from the puppy linux live cd:
That whole table is within 4GB of physical addressing. To get close to 4GB of physical ram requires more than 4GB of physical addressing, because a big chunk of the first 4GB of physical addresses are reserved for things other than ram.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Why size of swap space is 2x of RAM??? Why not 1.5x of RAM or 2.5x of RAM??? Saurav Ghosh Linux - Newbie 7 11-01-2011 03:49 AM
Possibility to create swap partition if extended partition is out of space? bhavisavla Linux - Newbie 1 09-12-2011 06:39 AM
Swap space did not function when physical memory ( RAM ) is full rabbit00 Linux - Newbie 19 07-06-2009 09:18 PM
LXer: Linux: Should You Use Twice the Amount of Ram as Swap Space? LXer Syndicated Linux News 1 11-19-2008 05:02 PM
SWAP space=2*RAM p_test Linux - Enterprise 3 04-27-2006 02:22 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 03:23 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration