LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 04-29-2013, 11:13 AM   #1
keif
Member
 
Registered: Apr 2013
Posts: 78

Rep: Reputation: Disabled
Format translation problem when emailing text file


Hello everyone,

I've been racking my brain on this and cannot seem to find anything on ol' trusty google. I'd appreciate any input.

Here is the problem:

I have a script that will ssh into several linux servers, run a couple commands and put the information into a file.

After its done it mails the file to myself.

The script works fine, since the text file looks good. The problem comes when I mail it to myself. When the text comes through in the mail, one of the lines is on top of the line before it.

Here's an example of how the input in the file looks:

hostname1
CHECK vision.error SIZE:
-rw-r--r-- 1 vision vision 689 Apr 29 11:44 vision.error
NUMBER OF LINES SHOWING fatal error:
0
=================------------------==================

Here's what it looks like when it is emailed to me:

hostname1
CHECK vision.error SIZE:
-rw-r--r-- 1 vision vision 689 Apr 29 11:44 vision.error NUMBER OF LINES SHOWING fatal error:
0
=================------------------==================

In the mailed format, lines 3 and 4 are on top of each other and they should be separated. They are separated in the text file, but for some reason mailing it puts them on top of each other.

I know this isn't a huge problem as far as data goes, but it's really bugging me! Anyone run into this sort of problem?

Thanks!

Keif
 
Old 04-29-2013, 12:04 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

And how can we figure out what's wrong with your code if you don't post it? We need to see what generated the mail command, and perhaps some details about the input as well.

Also, the text is in a file? Is this a temporary file that was generated by the script? If so, you might try running it through cat -A to see if there are any embedded non-printing characters involved.
 
Old 04-29-2013, 12:49 PM   #3
keif
Member
 
Registered: Apr 2013
Posts: 78

Original Poster
Rep: Reputation: Disabled
David, thank you for responding. Sorry about the lack of info. Will be sure to get everything you need in future posts.

Here is the code for the script:

Code:
#!/bin/sh

for STORE in `grep ^lf0 /etc/storelist | awk '{print $1}'`
do
echo $STORE >> /home/techs/lf-vision.error-large
echo CHECK vision.error SIZE: >> /home/techs/lf-vision.error-large
ssh vision@$STORE "ls -l data/status/vision.error" >> /home/techs/lf-vision.error-large
echo NUMBER OF LINES SHOWING fatal error: >> /home/techs/lf-vision.error-large
ssh vision@$STORE "grep -i 'with Fatal ERROR=' data/status/vision.error | wc -l" >> /home/techs/lf-vision.error-large
echo =================------------------================== >> /home/techs/lf-vision.error-large
done
mail -s "Check for Fatal Error Message in LF" myemail@hostname.com < /home/techs/lf-vision.error-large
Everything gets put into a file called lf-vision.error-large and the text format within this file looks correct, as shown here:

Code:
lf0002
CHECK vision.error SIZE:
-rw-r--r-- 1 vision vision 6893 Apr 29 11:44 data/status/vision.error
NUMBER OF LINES SHOWING fatal error:
0
=================------------------==================
The data below is copied from the actual email I receive with the data from the file lf-vision.error-large:

Code:
lf0002
CHECK vision.error SIZE:
-rw-r--r-- 1 vision vision 6893 Apr 29 11:44 data/status/vision.error NUMBER OF LINES SHOWING fatal error:
0
=================------------------==================
As you can see, line 4 is printing at the end of line 3 instead of printing below line 3.

Let me know if you need any additional information. Thanks again for looking.

Keif
 
Old 04-29-2013, 09:52 PM   #4
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,242

Rep: Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024
You could try David's suggestion of 'cat -A' to check that line for weird chars (or use od).

A quick fix might be to add an 'echo' in there to enforce a (blank) newline
Code:
ssh vision@$STORE "ls -l data/status/vision.error" >> /home/techs/lf-vision.error-large
echo >>/home/techs/lf-vision.error-large
echo NUMBER OF LINES SHOWING fatal error: >> /home/techs/lf-vision.error-large
 
1 members found this post helpful.
Old 04-30-2013, 07:36 AM   #5
keif
Member
 
Registered: Apr 2013
Posts: 78

Original Poster
Rep: Reputation: Disabled
Chris, adding the additional echo line seemed to give me a space between the lines so they are not on top of each other:

Code:
lf0002
CHECK vision.error SIZE:
-rw-r--r-- 1 vision vision 2273 Apr 30 07:31 data/status/vision.error

NUMBER OF LINES SHOWING fatal error:
0
=================------------------==================
Thanks for your assistance.

Keif
 
Old 05-05-2013, 06:45 AM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Thanks for posting the code. It would help a bit if you could format it better though, with proper indentations of sub-blocks and line spacing between logical sections.

Scripting With Style

Both of the outputs you posted show the same text, BTW. Is this an error on your part, or is it really that way?

But before we address this directly, I want to comment on the script itself.


First of all, does this have to be a POSIX-compatible /bin/sh script? If you can use bash or ksh instead, and their more advanced features, some things would be easier to code for.

The first, main error I noticed was the improper use of a for loop, along with a Useless Use Of Grep.

(Or actually, in this case, the awk command can be replaced with read, so it's really the unnecessary command here.)

$(..) is highly recommended over `..` too.

Finally, it's not a good idea hard-code data structures like filenames into a script. Set them first as variables at the top, or import them from elsewhere.

The script as I would write it:
Code:
#!/bin/sh

infile='/etc/storelist'
outfile='/home/techs/lf-vision.error-large'
mailadd=myemail@hostname.com

while read -r store _; do

    server="vision@$store"

    echo "CHECK vision.error SIZE"
    echo "$store"
    ssh "$server" "ls -l data/status/vision.error"

    echo "NUMBER OF LINES SHOWING fatal error:"
    ssh "$server" "grep -ic 'with Fatal ERROR=' data/status/vision.error"

    echo =================------------------================== >>

done <<INPUT >>"$outfile"
$( grep '^lf0' "$infile" )
INPUT

mail -s "Check for Fatal Error Message in LF" "$mailadd" < "$outfile"

exit 0
In order to code it for /bin/sh compatibility I used a here document for the while loop input. With bash we could use a process substitution instead. "The _" in the read command is a throwaway variable that takes the unwanted part of the line, so we get only the first field in the $store variable.

Notice also how you can redirect the entire output of the loop at once to the outfile. It would be even better not to use an external file at all, though, if you don't need it for other things. The script could easily be modified to store the output in a variable or array for later use. Or it could set it up as a function and it's output redirected directly into mail.

I notice that there's nothing removing or clearing the outfile after it's been used. Subsequent runs will include all the old data as well, unless you add a line to delete it.

And one final suggestion. If possible, how about combining the two ssh commands into one? There's no need to make two connections when one will do, although that would probably require modifying the output format.


Now to get back to the main problem. If we assume that its the main outfile that contains the improper lines, then the only possible explanation I can see within the script is that the ssh'd "ls -l" command isn't tacking on a trailing newline for some reason. But since you've already tried adding another echo, that doesn't seem to be the case.

So if that isn't the source of the error, and the outfile does contain the correctly formatted lines, then the problem must have something to do with the way mail processes the input. But that's a program that I don't have much experience with.

Last edited by David the H.; 05-05-2013 at 06:50 AM. Reason: minor
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Why is the text file format changing while transferring thru FTP ? apanimesh061 Programming 4 09-22-2011 02:56 PM
Need to convert pdf file to text format in linux 4.7 rahman2691 Linux - Newbie 6 12-27-2010 08:50 AM
i need to output a text file in proper format anurupr Linux - Newbie 2 03-04-2010 03:14 AM
how to format a text file to automate testing procedure Ashok_mittal Linux - Newbie 1 03-17-2008 08:39 AM
Problem in translation file PO aboakai Linux - Software 0 11-01-2004 07:20 AM


All times are GMT -5. The time now is 03:44 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration