LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 11-15-2008, 04:17 PM   #1
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Rep: Reputation: 15
using ed, sed to correct database


Okay, I dunno if I'm posting in the correct Forum but here it goes:

I have a huge tape library that has some unique attributes; all of the tracks are separated into folders by date (mostly)...each folder is labeled in this syntax by default:

LOCATION-YEAR-MN-DY- Subject -SPK (The last three letters are the initials of the speaker on the track) I have about a thousand of these folders and there are more to come...I want to make a file with a list of the directories and replace the Location-YEAR-MN-DY syntax with a more conventional syntax of DY-MN-YEAR so that I can redirect to an even larger flat file (excel) and compare data...so the creating a file part is simple (a simple ls redirection) but coming up with the syntax for sed is eluding me...can anyone help...here is a sample of the file:


Code:
1984-04-20- Ordanation!!!!!-BLH
1999-08-08- The Holy Ghost! - BCH
Albuquerque-2004-07-10- Being a Dedicated Christian -BLH
Baton Rouge-2003-4-13 -Unity_Building the Body-BLJ
Berkeley-06-03- Revelations 138 Worlds -BLH
Berkeley-1976-04-25- Ressurection -BLH
Berkeley-1976-11-25- Fulfillment of Pr
Berkeley-1976-11-25- Time -BLH
Berkeley-1978-11-25 Early Church -BLH
Berkeley-1983 - 10-21 -Attibutes of Christ - BLH
Berkeley-1983-02-02- Perfection Overcoming -BLH
Berkeley-1983-02-04- Perfection _ Brotherly Love - BLH
Berkeley-1983-06-19 - Making the Bride - BCH
Berkeley-1983-10-21- Jesus Christ - BLH
Berkeley-1983-10-23- Washing yourself from the World - BCH
Berkeley-1983-11-04- Trying to Build the body - BJ
Berkeley-1983-11-04- Trying to Build the body - BJS
Berkeley-1983-11-06- Fireside about importance of family -BS
Berkeley-1983-12-09- Daniel 8 -BLH
Berkeley-1983-12-09- Q&A - Daniel 8 -BLH
Berkeley-1983-12-23- Restoration of the Church - BLH
 
Old 11-15-2008, 06:17 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,188

Rep: Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131
That is *UGLY* data - not consistent at all. Some leading zeroes missing, imbedded blanks in the date sometimes - one line doesn't even have the year. Try this to get you started
Code:
 sed -r 's/(.*)([0-9]{4})\s*-\s*([0-9]{1,2})\s*-\s*([0-9]{1,2})(.*)/\4-\3-\2\5/' junk.txt
 
Old 11-15-2008, 06:19 PM   #3
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
Truncate, reformat?

Couldn't I truncate all of the spaces and such before doing the search?

I will run that command and get back to you with my results.
 
Old 11-15-2008, 06:40 PM   #4
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
ok

Ok I just ran the command and the good news is there were no errors...the bad news is that is didn't make a single change....when I cat the file the format remained the same...
Here is what I typed "directory" being the name of the file that contains the directory listing..

Code:
$ sed -r 's/ (.*) ([0-9] {4} )\s*-\s*([0-9]{1,2})\s*-\s*([0-9]{1,2})(.*)/\4-\3-\2\5/' directory
 
Old 11-15-2008, 06:48 PM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,188

Rep: Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131
Write it out to a new file "... > newfile.txt"
 
Old 11-15-2008, 07:22 PM   #6
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
hmmm same thing...I redirected output to another file called directory2:

Code:
$ sed -r 's/ (.*) ([0-9] {4} )\s*-\s*([0-9]{1,2})\s*-\s*([0-9]{1,2})(.*)/\4-\3-
\2\5/' directory > directory2
can I forward you a copy of the file to try the command on it yourself?
 
Old 11-15-2008, 07:27 PM   #7
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,188

Rep: Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131
I would suggest you try the command I posted - unmodified.
 
Old 11-15-2008, 07:40 PM   #8
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
Smile I stand corrected

You are correct sir. Now I'll just spend some time decrypting your syntax so that I might replicate it in the future. Thank you very much!!
 
Old 11-15-2008, 07:42 PM   #9
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
one more thing

How might I append the location to the end (since the former command removed the location?)
 
Old 11-15-2008, 08:58 PM   #10
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,188

Rep: Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131
There's plenty of ideas for you to look at in that command - it does what you asked for initially. The other data is still available if you didn't destroy the source. I'll leave it as an exercise for you to finish it off. There is good doco on the sed.sf.net site.
 
Old 11-15-2008, 09:23 PM   #11
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
sed command clarification

Quote:
Originally Posted by syg00 View Post
That is *UGLY* data - not consistent at all. Some leading zeroes missing, imbedded blanks in the date sometimes - one line doesn't even have the year. Try this to get you started
Code:
 sed -r 's/(.*)([0-9]{4})\s*-\s*([0-9]{1,2})\s*-\s*([0-9]{1,2})(.*)/\4-\3-\2\5/' junk.txt
Okay I'm going to attempt to ask an educated question about what you've done with this command...its very impressive to me that it didn't take you like 3 days to figure out that expression...here it goes:

Question 1: At the beginning of your expression you used sed -r 's/
Why would you need the '-r' option...from the man pages it says its for using extended regular expressions in the script. What extended expressions did you use?

Question 2: When you cached the output of ([0-9]{4}) in what memory block did you store it in? I dont see a \1 or \2 until later on in the expression but I do see several instances of \s* what does that do?

Thanks for the clarification.
 
Old 11-15-2008, 10:05 PM   #12
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,188

Rep: Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131Reputation: 4131
1) Supposedly the "-r" isn't required, but often the substitution of subexpressions on RHS is rejected without it. Try it and see. Becomes habit after a while.
2) \s* is any number (zero or more) whitespace characters - tabs, blanks. Very handy when you aren't sure what the actual "space" is. Now about that "-r" ...
 
Old 11-15-2008, 10:08 PM   #13
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
Thanks!!

Quote:
Originally Posted by syg00 View Post
There's plenty of ideas for you to look at in that command - it does what you asked for initially. The other data is still available if you didn't destroy the source. I'll leave it as an exercise for you to finish it off. There is good doco on the sed.sf.net site.
Thank you for the website reference...I'll begin my studying. Thanks for your help.
 
Old 12-02-2008, 03:51 PM   #14
ifeatu
Member
 
Registered: Sep 2008
Distribution: Fedora 9
Posts: 68

Original Poster
Rep: Reputation: 15
I purchased the O'Reilly Book by J. Freidl on RegEx (ISBN: 9780596528126) I have no idea how I got along this far without this book...thanks for your advice....I also joined a sed yahoo group.

Thanks!
 
Old 12-02-2008, 04:03 PM   #15
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
Holy cow!!! I thought I was a SED fanatic.

Another good reference:
http://www.grymoire.com/Unix/
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How can i copt the MYSQL Database to a other server (Database) halvorls Linux - Server 3 07-27-2008 09:19 PM
Database Programming (database to database transaction) johncsl82 Programming 7 02-02-2007 08:20 AM
sed script to read only columns 4 to 6 in output database cranium2004 Programming 10 02-28-2006 07:20 AM
Insert character into a line with sed? & variables in sed? jago25_98 Programming 5 03-11-2004 06:12 AM
Is this the correct partition strategy for a database? cstovall Linux - General 1 03-03-2003 03:04 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:12 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration