LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 02-10-2013, 05:07 PM   #1
zaphodpr
LQ Newbie
 
Registered: Feb 2013
Posts: 3

Rep: Reputation: Disabled
Hi!!! what is wrong with this??


This script is supposed to grab the last line of text for all the files in a directory and create a file with those lines if it were to work at all... some lines that start with # do not work and I need to keep pressing enter for it to work when it should be automatic... Any help???

#!/usr/bin/bash
# touch < date "+%Y-%m-%d-%H-%M-%S"

NOW=$(date "+%Y-%m-%d-%H-%M-%S")
LOGFILE="log-$NOW.log"
touch $LOGFILE
FILES=/Users/zaphod/Desktop/me/data/*.dat
for f in $FILES
#TODAY="Today is $NOW"
#echo $TODAY
#cat $TODAY >> $LOGFILE
do
echo "processing $f file..."
# echo -e "\n"
# tail -n 1 $f
tail -n 1 $f
cat -u | (tail -n 1 -q $f) > "\n" >> $LOGFILE
#echo -e "\n"
done
 
Old 02-10-2013, 06:50 PM   #2
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,225

Rep: Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021
1. use 'echo' to output strings
2. use cat to output files
3.
Code:
# replace this
tail -n 1 $f
cat -u | (tail -n 1 -q $f) > "\n" >> $LOGFILE 

# with
tail -n 1 $f >>$LOGFILE
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

4. To see what the parser is doing, amend top of script
Code:
#!/usr/bin/bash
set -xv
5. incidentally, on my system its /bin/bash; you may want to double check your system.

6. it would help to know exactly what OS, Sw you have

Code:
cat /etc/*release*
uname -a
bash --version
 
Old 02-10-2013, 07:25 PM   #3
zaphodpr
LQ Newbie
 
Registered: Feb 2013
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks Chrism01. This code is meant to run on Debian. I ran out of time on the weekend and had to change the paths so it would work on OS X so I could work on my laptop during weekdays. I have done everything I know to make it OS neutral. But, it shows I don't know crap; debian bash is version 4.2 but OS X version is 3.2.
 
Old 02-10-2013, 07:35 PM   #4
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,225

Rep: Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021
Well, hope that helped.
Have a read of those links; they're very good
 
1 members found this post helpful.
Old 02-11-2013, 11:38 AM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
To start with, please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

Let me go line-by-line, so I can point out some things.

Code:
# touch < date "+%Y-%m-%d-%H-%M-%S"
Yes, it's a comment, but the code in it is wrong. You can't directly run multiple commands together like that. You have to use a command substitution. As it is, "date" would be treated as a filename for redirection instead. But then, touch doesn't accept strings from stdin by default either.

Code:
LOGFILE="log-$NOW.log"
touch $LOGFILE
QUOTE ALL OF YOUR VARIABLE EXPANSIONS. You should never leave the quotes off a parameter expansion unless you explicitly want the resulting string to be word-split by the shell (globbing patterns are also expanded). This is a vitally important concept in scripting, so train yourself to do it correctly now. You can learn about the exceptions later.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Now, in this specific case it doesn't cause problems because the expanded string doesn't have any whitespace or globbing characters. But it's a good idea to always do it properly anyway.

And since environment variables are generally all upper-case, it's recommended practice to keep your own user variables in lower-case or mixed-case to help differentiate them.

Code:
FILES=/Users/zaphod/Desktop/me/data/*.dat
for f in $FILES
do
This is ok, as the unquoted $FILES variable will glob expand properly. But as I just said, it's better to keep variables quoted whenever possible. A better technique would be to use an array (assuming bash or another shell that supports them).

Code:
files=( /Users/zaphod/Desktop/me/data/*.dat )
for f in "${files[@]}"; do
Or just use the globbing pattern directly after for.

You might also consider not hard-coding the file path like that, but instead cd'ing into that directory at the beginning of the script, so you could use a simple "for f in *.dat".

Note too that many scripters feel that it's better to place the "do/then" keywords on the same line as the "for/while/until/if" keywords, as they are not separate commands but the closing indicators for the test block, paired directly with the opening keyword. Putting them together on one line thus more clearly separates the outside block from the inside block.


Code:
# echo -e "\n"
Again, not really a problem, but this seems to me like a bit of pointless complexity, and a slightly non-transparent one. I'd personally just use "echo; echo", or "printf '\n\n'" if I wanted to print two blank lines (learn about printf here).

Code:
cat -u | (tail -n 1 -q $f) > "\n" >> $LOGFILE
I can't understand what this is supposed to be doing. The initial cat command (-u is unrecognized in the gnu version) would just sit there waiting for input from stdin. If it ever got some, then it would try to pipe it into tail (which is also running in a pointless subshell), but tail is already reading from a file, and will probably ignore stdin. Finally, it tries to output it's results to both "\n" and to "$LOGFILE"? That doesn't work either. redirections) are processed left-to-right at the beginning of command parsing, so the second one simply overrides the first one. In any case it certainly doesn't add a newline to the output, as you seem to think it does.

chrism01 has already posted how to run tail correctly. But if you ever need to add a newline to a file, just run "echo >> filename" after the main command.

Finally, beware the Useless Use Of Cat!
 
Old 02-11-2013, 05:35 PM   #6
zaphodpr
LQ Newbie
 
Registered: Feb 2013
Posts: 3

Original Poster
Rep: Reputation: Disabled
Smile Thanks!!!

This is a working and more elegant version ( I think) of the script you guys helped me out with:
Code:
 #!/bin/bash
#set -xv
NOW=$(date "+%Y-%m-%d-%H-%M-%S")
LOGFILE="log-$NOW.log"
touch $LOGFILE
FILES=/Users/zaphod/Desktop/me/data/*.dat
echo $LOGFILE
echo $NOW
printf "Today is: " >> $LOGFILE
printf $NOW >> $LOGFILE
#printf date >> $LOGFILE
printf "\n" >> $LOGFILE

for f in $FILES
   do
      echo "processing $f file..."
      tail -n 1 $f >> $LOGFILE
      printf "\n" >> $LOGFILE
done

printf "Pending Jobs: \n" >> $LOGFILE
more /Users/zaphod/Desktop/me/data/errands >> $LOGFILE
printf "\n" >> $LOGFILE
I still need to fix the file locations issue, but I guess I can create an environment variable so it works both on Debian and OS X. I am grateful, and only have one Issue... I can't seem to find a way to add a human readable date string at the beginning of the file. Can that be done? Thank you very much guys!!! You mede me look like i was trying to light a cigarette with a damaged flame thrower
 
Old 02-17-2013, 11:50 AM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Quote:
Originally Posted by zaphodpr View Post
I can't seem to find a way to add a human readable date string at the beginning of the file. Can that be done?
To insert the output of a command to the beginning of a file, or at any location other than the end, you generally have to use an external command like sed or ed.

Code:
printf '%s\n' 'H' '0a' "$NOW" '.' 'w' | ed -s d_file.txt
How do I prepend a text to a file (the opposite of >>)?
http://mywiki.wooledge.org/BashFAQ/090

How can I redirect the output of 'time' to a variable or file?
http://mywiki.wooledge.org/BashFAQ/032

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)


PS: Now you're really using printf incorrectly. The idea behind printf is that you first define a formatting string, and this format is then used to print out the subsequent arguments. The first argument should thus always be the format string, which should generally contain at least one "%" placeholder. This is followed by one or more argument strings that will replace those placeholders. printf will then loop until all arguments are consumed. And don't forget that, unlike echo, printf does not include any newlines unless you explicitly include them.

If you don't need to do fancy formatting on repeated strings, just use echo.

Code:
#echo simply prints all arguments, followed by a newline.
echo "Today is: $NOW" >> "$LOGFILE"

# The same command using printf:
printf 'Today is %s\n' "$NOW" >> "$LOGFILE"
An argument could be made that I was also technically using printf incorrectly in my last post, but I justify it by the fact that you wanted to print two newlines in a row. Since printf always prints the format string at least once, and automatically expands any backslash escapes in it, this makes it slightly cleaner than using two echos.

See the link I gave before for more details.


And please don't forget what I said before about quoting variables!

Last edited by David the H.; 02-17-2013 at 11:59 AM. Reason: tag fix
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Wrong disk free space (VERY wrong) macrojd Linux - Server 12 05-17-2009 02:14 AM
rsync uses wrong directory, has wrong size but right file count? brianpbarnes Linux - Software 1 02-23-2009 05:48 PM
Fortran - G95 - Wrong Syntax or wrong compiler settings laucian Programming 1 03-21-2008 10:18 AM
2domain but backingup the wrong one .. what am i doing wrong salimmeethoo Linux - Server 3 10-17-2007 10:43 AM
What is wrong with reiserfs? wrong free space mesh2005 Linux - General 1 05-03-2007 07:21 AM


All times are GMT -5. The time now is 10:24 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration