Trying to figure out how to get started with scripting...

dave247 · 10-20-2008, 09:02 PM

Hi. I am just getting into scripting. I came into a situation where I need to go through several hundred files on a Linux system and find a couple specific bits of information from within each file. All the files have pretty much identical content except for a view data values on the same two lines in each file. I want to make a script that can go through them all and get each data value so I can manipulate them.

I know this is kind of a broad question, but I am really just trying to figure out where to get started. I am a bit of a Linux and programming novice, so I think this is going to be something difficult, but I do know the basics of Linux and of what a shell script is and how it works, etc. Now I am reading about bash scripting from several different websites, but it seems like I might have to go through a lot of stuff and study for a few weeks before I can even accomplish something. I need to get this done much sooner than that.

Can anyone refer me to a good script site where I can dissect already made similar scripts or maybe a page that talks about file reading and manipulation scripts?

Any other kind of help is appreciated...

EDIT: I mean, what kind of commands would be wise to put in this script? Find? AWK?

darthaxul · 10-21-2008, 12:30 AM

buy a good book
http://oreilly.com/pub/topic/linux
or try
http://python.org/
even nasa uses that lol

dave247 · 10-21-2008, 12:38 AM

Quote:

Originally Posted by darthaxul

buy a good book
http://oreilly.com/pub/topic/linux
or try
http://python.org/
even nasa uses that lol

i already bought one and its in my hands now; called "Mastering Unix Shell Scripting"

but this is going to take me forever to get through this $h1T

chrism01 · 10-21-2008, 01:35 AM

I reckon these 3 are good:

http://rute.2038bug.com/index.html.gz : Linux from the cmd line
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html : what it says

http://www.tldp.org/LDP/abs/html/ : advanced shell (bash)

If you don't know where the files are, look up the 'find' cmd

man find

to check the contents lookup 'grep'

man grep

You'll prob need a loop:

Code:

for file in `ls`
do
    result=`grep <your-string-here> $file`
    if [[ $? -eq 0 ]]
    then
        # matched, do something eg echo result to caller
        echo $result
    fi
done

HTH

dave247 · 10-21-2008, 08:30 AM

chrism, thanks but I've already been studying those last two links you gave...

I do know where the files are.... i need to figure out what commands to use to search through them all to pull out two certain pieces of information. Dont know how

ErV · 10-21-2008, 08:45 AM

Quote:

Originally Posted by dave247

Hi. I am just getting into scripting. I came into a situation where I need to go through several hundred files on a Linux system and find a couple specific bits of information from within each file. All the files have pretty much identical content except for a view data values on the same two lines in each file. I want to make a script that can go through them all and get each data value so I can manipulate them.

Can be done in python or shell (and probably dozen other languages I never used).
In python see documentation about "glob" command. (also if you are total newbie read about arrays, dictionaries, reading from files, and comparing strings. All this information is available in python tutorial)
In shell, you'll need "find" command.

The best way to start learning scripting is analyzing existing scripts (assuming you have several books nearby and some kind of programming/computer knowledge). See what scripts your system has, try to understand at least part of them.

openSauce · 10-21-2008, 01:17 PM

awk is very good for this sort of thing. If you were an experienced programmer, I'd say definitely go for that, you can pick up all you need in a couple of hours from the man page.

Since you say you're a novice I'm not sure how hard it would be; python is designed to be easy for novices to pick up, so you might want to start there. Bash syntax is quite disgusting IMO, although still worth knowing if you use Linux a lot. I'm slowly getting used to it I guess.

Anyway, to get you started, this command will go through all files in the current directory and print the 2nd and 3rd words of any line containing the text "THIS-LINE" (with words separated by spaces):

Code:

awk '/THIS-LINE/ {print $2, $3}' *

Alternatively, this command will print the 2nd and 3rd words of the 5th and 6th lines of each file:

Code:

awk '{if (FNR==5 || FNR==6) print $2, $3}' *

It sounds like that's along the lines of what you want. You can find awk tutorials online, and the manpage is good. The syntax of awk programs is much like C or Java, so looking at sample programs in those languages might help if you're stuck on how to write a complex if statement, or a for loop. Hope that helps!

Edit to add: once you've got the output in the format you want, redirect it to a file or pipe it to another program - the tutorials you've been looking at will tell you how

pixellany · 10-21-2008, 01:34 PM

Quote:

Originally Posted by dave247

chrism, thanks but I've already been studying those last two links you gave...

I do know where the files are.... i need to figure out what commands to use to search through them all to pull out two certain pieces of information. Dont know how

Just one of many methods:
With all the files in the same directory:

for file in `ls`; do
echo $file
grep keyword $file
done

this prints the name of each file, and the line containing "keyword" (if found). Look at the -A, -B, -C flags for GREP for more options.