LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   End-of-line Characters missing from last line of md5 file. Md5sum fails (http://www.linuxquestions.org/questions/linux-general-1/end-of-line-characters-missing-from-last-line-of-md5-file-md5sum-fails-736546/)

mehorter 06-29-2009 08:18 PM

End-of-line Characters missing from last line of md5 file. Md5sum fails
 
I have a number of md5 files that were either generated on windows machines or transfered in binary mode (bit for bit). As a result the end-of-line characters are formated with CRLF.

Code:

cat -e windows.md5
d4c638e28b1990c55b79e5249cf33038 *testfile_01.ext^M$
9e15845bdae8c1d1699dade90b73c744 *testfile_02.ext

When I translate them with
Code:

tr -d '\r'  <windows.md5> linux.md5
the CR is removed
Code:

cat -e linux.md5
d4c638e28b1990c55b79e5249cf33038 *testfile_01.ext$
9e15845bdae8c1d1699dade90b73c744 *testfile_02.ext

and I can use
Code:

md5sum -c linux.md5
with success except for the last file/line to be checked.

My problem is that the last line in the file never had any end-of-line characters to begin with.

As a result, md5sum -c linux.md5 returns an error for the last line.
Code:

md5sum -c linux.md5
testfile_01.ext: OK
md5sum: testfile_02.ext : No such file or directory
testfile_02.ext : FAILED open or read
md5sum: WARNING: 1 of 2 listed files could not be read

If I open the linux.md5 file with EMACS, delete the last "blank" characters of the file then hit return, then save that I find the last line is fixed
Code:

9e15845bdae8c1d1699dade90b73c744 *testfile_02.ext$
and will be used by md5sum correctly.

I would like to be able to script the fix and not have to manually go through every md5 file by hand. It seems like an easy problem but it has got me vexed...

I have tried appending to the file:
Code:

cat >> linux.md5
Blah
^D

but that only returns
Code:

d4c638e28b1990c55b79e5249cf33038 *testfile_01.ext$
9e15845bdae8c1d1699dade90b73c744 *testfile_02.ext $
Blah$

Please notice the space between the "testfile_02.ext" and "$".
This returns the same error.

If I could only insert the correct end-of-line character for the last line I think I would be fixed.

Any suggestions?

GrapefruiTgirl 06-29-2009 09:04 PM

To get you started, there is a tool/script already available, and included with many *nix systems, called 'dos2unix' and it has a partner tool called 'unix2dos' and it does the end-of-line conversions on text files. Check Google for it if it is not on your machine.

As to the missing end-of-line on the very last line, I'm not sure if dos2unix would insert one (I haven't used the tool myself) but it wouldn't be too much of a big leap to have a final newline appended to a file that doesn't have one.

Sasha

mehorter 06-29-2009 09:14 PM

Quote:

Originally Posted by GrapefruiTgirl (Post 3590802)
To get you started, there is a tool/script already available, and included with many *nix systems, called 'dos2unix' and it has a partner tool called 'unix2dos' and it does the end-of-line conversions on text files. Check Google for it if it is not on your machine.

As to the missing end-of-line on the very last line, I'm not sure if dos2unix would insert one (I haven't used the tool myself) but it wouldn't be too much of a big leap to have a final newline appended to a file that doesn't have one.

Sasha

Thanks,

I do have dos2unix installed and it gives the same result as
Code:

tr -d '\n' <oldfile> newfile
Both methods fail to leave any kind of EOL character on the final line.

Please make the leap and show me how to have a final newline appended to a file that doesn't have one!

I might also mention that I get the same results if I use vi and use 'set fileform=unix'. Same thing again when i use
Code:

sed -e 's/\r$//' old.md5 > new.md5
Seems like a job for sed or awk but I'm vexed..

GrapefruiTgirl 06-29-2009 09:25 PM

sh# awk '{ printf "\n" >> $0 }' md5sum_file.txt

sh# echo >> md5sums_file.txt

There are a couple methods I just quickly grabbed off the 'net :) but haven't tested.
In the case of a file that already has a newline, a second one shouldn't hurt (I don't think it will :scratch:)

Sasha

mehorter 06-29-2009 09:52 PM

Quote:

Originally Posted by GrapefruiTgirl (Post 3590823)
sh# awk '{ printf "\n" >> $0 }' md5sum_file.txt

sh# echo >> md5sums_file.txt

There are a couple methods I just quickly grabbed off the 'net :) but haven't tested.
In the case of a file that already has a newline, a second one shouldn't hurt (I don't think it will :scratch:)

Sasha

Allright!

Both methods work great!
Code:

tr -d '\r' <windows.md5> linux.md5; awk '{ printf "\n" >> $0 }' linux.md5
Code:

tr -d '\r' <windows.md5> linux.md5; echo >> linux.md5
Both methods remove CRLF from all lies and leave all lines with a LF (line feed).

The 2nd method using echo does insert an extra Character but that doesn't seem to effect the function of md5sum.

Thank-you!

GrapefruiTgirl 06-29-2009 09:56 PM

No problem; I learned something myself too: a simple AWK function. I have spent much time learning what I can of GAWK, but AWK has stuff too that can be handy-- if only there weren't SO MANY permutations of AWK/GAWK/SED they would be easier to get a handle on.

Cheers,
Sasha


All times are GMT -5. The time now is 08:36 PM.