LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 05-15-2008, 06:47 AM   #1
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Rep: Reputation: 30
how to set sed to show random md5 hash!


Hello, does anyone know how to make sed get the md5 checksum hash from a file.

I have some files that have random md5 checksum hash on it content, trying to figure out how i can set sed to know the md5 hash between all the lines and words in the files.

md5 is a 32 character string of a random hexadecimal numbers [0-9][a-f].

i tried to set identifier in sed to show the hash after it, but it also show the worlds in the same line after the hash.

any idea ?

Last edited by Dr_Death_UAE; 05-15-2008 at 06:50 AM.
 
Old 05-15-2008, 06:54 AM   #2
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
You will have to show what you have so far, or a sample line before someone can provide much help.

You probably want to use the form:
sed '/<md5pattern>/s/.*\(<md5pattern>\).*/\1/' file
to select the line with the md5sum and to only output the match.
 
Old 05-15-2008, 07:09 AM   #3
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
no jschiwal, i dont have some thing to match.

some of the files they have "md5sum=" before the hash, so i can get the hash, but it show also words after the hash for example:

file:
Quote:
filename=test, md5sum=764efa883dda1e11db47671c4a3bbd9e, size=402KB
sed command:
Quote:
sed 's/.*md5sum=[0-9][a-f]*//' file
will show:
Quote:
md5sum=764efa883dda1e11db47671c4a3bbd9e, size=402KB
i was wounder how to set sed to know the hash base on its length 32 char, and it is have [0-9][a-f]

thanks
 
Old 05-15-2008, 07:18 AM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Why not gawk?
Code:
gawk '/md5sum=/{print substr($0,index($0,"md5sum=")+7,32)}' file
or if you want to print the "md5sum=" part, too
Code:
gawk '/md5sum=/{print substr($0,index($0,"md5sum="),39)}' file
 
Old 05-15-2008, 07:31 AM   #5
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
thanks colucix, awk work fine, is there a way to get the hash without any identifier, i mean if the "md5sum=" is a variable and the hash also a variable. not all of the files have "md5sum=".

thanks again
 
Old 05-15-2008, 08:36 AM   #6
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
So you want to match any sequence of 32 characters matching digits or letter from a to f (hexadecimal digits), right? Something like this should work
Code:
gawk --posix '{idx = match($0,/[0-9a-f]{32}/); if ( RLENGTH == 32 ) print substr($0,idx,RLENGTH)}' file
Please note that you have to specify the --posix option of awk to let it recognize the braced interval expression {32} in the regular expression, which is disabled by default.
 
Old 05-15-2008, 09:44 AM   #7
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
Thanks colucix, it didn't work. but i got the idea. thanks a gain.


Quote:
h4x0r# echo "test" | md5 > test
h4x0r# cat test
d8e8fca2dc0f896fd7cb4cb0031ba249
h4x0r# awk --posix '{idx = match($0,/[0-9a-f]{32}/); if ( RLENGTH == 32 ) print substr($0,idx,RLENGTH)}' test
h4x0r#
 
Old 05-15-2008, 11:16 AM   #8
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Strange... it works on my system. Which version of awk do you have?
 
Old 05-15-2008, 02:39 PM   #9
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
h4x0r# awk --version
awk version 20070501 (FreeBSD)
 
Old 05-15-2008, 03:13 PM   #10
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Hmmm... once I read about the FreeBSD version of awk not supporting extended regular expressions and in particular the interval expression I used in the command above. I don't know much about FreeBSD, but as alternative you can try something like
Code:
grep -E -o '[0-9a-f]{32}' file
I'm not sure if grep on FreeBSD supports the -o option, to print the matching part only.
 
Old 05-15-2008, 03:22 PM   #11
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
grep work fine thanks dude, is there a difference between awk and gawk?
 
Old 05-15-2008, 03:34 PM   #12
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Yes. A lot of differences, in the sense that the GNU awk (gawk) has introduced new advanced features not available in standard unix's awk. Portability of your awk code is an issue when you are used to the GNU extension of awk.
 
Old 05-15-2008, 03:49 PM   #13
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
aha, thanks for the info brother.
 
Old 05-16-2008, 06:48 AM   #14
Dr_Death_UAE
Member
 
Registered: Jul 2005
Location: U.A.E
Distribution: FreeBSD,Fedora,Solaris,AIX
Posts: 168

Original Poster
Rep: Reputation: 30
i install gawk on my FreeBSD, the posix work fine with it.

Quote:
h4x0r# gawk --version
GNU Awk 3.0.6 + multi-byte extension 1.15
h4x0r# cat md5test
filename=test, md5sum=764efa883dda1e11db47671c4a3bbd9e, size=402KB
h4x0r#
h4x0r# gawk --posix '{idx = match($0,/[0-9a-f]{32}/); if ( RLENGTH == 32 ) print substr($0,idx,RLENGTH)}' md5test
764efa883dda1e11db47671c4a3bbd9e
h4x0r#
thanks
 
Old 05-16-2008, 07:55 AM   #15
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Good to know! You're welcome!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
md5 hash files mattydee Linux - Security 3 05-07-2008 06:42 PM
need help unpacking hmac-md5 hash into md5 hash lynx5 Programming 3 02-02-2008 04:06 PM
gui md5 hash tool monohouse Linux - Software 1 05-17-2005 06:00 PM
checking MD5 hash issey Linux - Newbie 4 03-31-2005 01:15 AM
MD5 hash value dominant Linux - Newbie 2 05-07-2004 03:52 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:20 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration