LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 12-18-2011, 08:29 AM   #1
rng
Member
 
Registered: Aug 2011
Posts: 747

Rep: Reputation: 23
sed or awk for text file manipulations


I want to learn an application for manipulating text files. Which is better in terms of features and ease of learning: awk or sed (or something else)?
 
Old 12-18-2011, 09:18 AM   #2
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,110

Rep: Reputation: 810Reputation: 810Reputation: 810Reputation: 810Reputation: 810Reputation: 810Reputation: 810
More likely, you'd want to learn both -- sed is the streaming editor where AWK is a programming language; sed can alter a file, AWK can make decisions depending upon conditions (and do arithmetic and other handy things).

sed is quite useful when you have either a large number of files or extremely large size files that you need to "clean up" or alter in some way in a stream (files are not loaded into memory, sed works on a line at a time in a pipeline). This allows multiple edits on every line and goes quick like a bunny.

AWK, on the other hand, is used to write small programs that can do large jobs quickly and efficiently.

Both of them are well worth your time to become comfortable with.

Hope this helps some.
 
1 members found this post helpful.
Old 12-18-2011, 12:06 PM   #3
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655
Awk is a natural for text organized in records of fields. If you are manipulating something like a mailing list or phone list, look at using awk first.
 
1 members found this post helpful.
Old 12-18-2011, 03:36 PM   #4
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 147Reputation: 147
Both are worthwhile, but if I could only choose one I'd make it AWK. AWK programs tend to be easier for me to read.

AWK has features and constructs common to high level programming languages. If you already know any C-like language, AWK should not be difficult to learn. Sed uses a language similar to GNU Ed, meaning single-symbol commands and a heavy reliance on regular expressions.

AWK sees use for complex tasks, such a Wiki software. Sed mostly sees use for simple substitutions and deletions. People have done some very cool things in both languages though. I seem to recall that someone wrote an RPN calculator in Sed.

Here are some examples of specific programs in AWK and Sed. Fields are separated by single `\t' (tab) characters. Additions, elaborations and corrections welcome.

Output entire file
Code:
test$ awk '{print}' birthday-cake.txt
Janet   June    chocolate
Ken     June    chocolate
Jeff    November        vanilla
Dan     January vanilla
test$ sed '#' birthday-cake.txt
Janet   June    chocolate
Ken     June    chocolate
Jeff    November        vanilla
Dan     January vanilla
test$
Find lines containing specific text
Code:
test$ awk '/November/' birthday-cake.txt
Jeff    November        vanilla
test$ sed '/November/!d' birthday-cake.txt
Jeff    November        vanilla
test$
Replace specific text in specific field
Code:
test$ awk 'BEGIN {OFS = "\t"}; $3 == "vanilla" {$3 = "strawberry"}; {print}' birthday-cake.txt
Janet   June    chocolate
Ken     June    chocolate
Jeff    November        strawberry
Dan     January strawberry
test$ sed 's/^\([^'$'\t''][^'$'\t'']*'$'\t''[^'$'\t''][^'$'\t'']*'$'\t''\)vanilla/\1strawberry/' birthday-cake.txt
Janet   June    chocolate
Ken     June    chocolate
Jeff    November        strawberry
Dan     January strawberry
test$
Don't let that last one scare you. The $'\t' bit is a Bash-ism for an explicit `\t' (tab) character. I had to make sure that the substitution occurred in the third field only, so it was necessary to count the tab characters between fields. My Sed-fu is weak; surely there is a better way.

I recommend reading the manuals for each program, and then decide for yourself which one seems best for the task at hand. Here is the manual for GNU Sed.

Most GNU/Linux systems probably have Gawk or Mawk, though there are other implementations available. It is important to know which implementation you have because they are not all the same. The command awk -Wversion should tell you which AWK you have. I recommend reading the GNU Awk User's Guide in addition to the manual for your AWK, as it is the most complete reference to the laguage I'm aware of.

HTH
 
Old 12-18-2011, 09:10 PM   #5
rng
Member
 
Registered: Aug 2011
Posts: 747

Original Poster
Rep: Reputation: 23
Thanks for explanations.
 
  


Reply

Tags
awk, sed


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how do you replace text between two words in a whole file not just 1 line w/ sed/awk lityit Programming 5 11-04-2011 01:04 AM
split very large 200mb text file by every N lines (sed/awk fails) doug23 Programming 8 08-10-2009 07:08 PM
Need help stripping statement from text file, ksh: sed awk? austin881 Programming 7 07-13-2009 05:31 PM
Manipulating Text File with awk or sed kushalkoolwal Programming 2 09-10-2008 08:35 PM
Help with a script to edit text file (awk? sed?) rickh Linux - Newbie 8 04-21-2005 09:24 PM


All times are GMT -5. The time now is 03:42 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration