Which one is efficient cut cmd or using awk
Hi,
You may find this bit silly. But I would like to understand it at the core level. I have a following line. Code:
line=this,forum,is,for,all,programming,questions Code:
ThirdField=$(echo $line|cut -d',' -f3) Which one is efficient(using cut cmd or awk). Which one works better in all situations. Right now, i can't think of any situation. However, I would like to know from all your experince that Have you find any difference. Is there any basic guidelines to write a shell script on efficiency part. Thanks in advance. Cheers, Suresh |
In my opinion, in this example, they are the same. You can check by using 'time' command. 'cut' is use when separator is colons, commas... while 'awk' is use when columns are separated by a varying number of spaces.
|
You can use the "time" command to benchmark the results. For example:
Quote:
'Hope that helps .. PSM |
You cannot get reliable timing doing something once. I wrapped them in a for loop to repeat them 1000 times, and then did 3 tests for each (to get a handle on the consistency). Cut averaged 18.735s for the thousand, while awk averaged 20.079s, which is 7% longer. A small difference; if one is seeking to improve performance, there are likely to be bigger gains elsewhere. (Starting with not using shell scripting!)
|
Last time I did a similar comparison, I compared a 10,000 iteration loop operating on a long silly string.
cut was faster than tr; tr was faster than sed; sed was faster than awk. cut and tr were very close. sed significantly faster than awk, doing substring replacements. FWIW |
My hunch is that most of the time goes to spawning subshells
Code:
line=this,forum,is,for,all,programming,questions |
Quote:
Quote:
Quote:
|
Quote:
|
Quote:
|
Quote:
Obviously awk can do many things cut cannot. But for those things cut can do, I expect it will generally be a little faster than awk. Of course, 'efficiency' can also refer to the programming/scripting stage. But that cannot be measured objectively, since as mentioned it depends on personal familiarity with the tools. |
Thankyou guys for all your valuable inputs on the efficiency part.
So, Can I conclude that " In my above scenario where I need to get the 3rd field from a record, CUT is better option than AWK. However, AWK is better in terms of options and flexibility" It really helped me coz, right now I need to loop through a file which has over 1 million records and get the 3rd field from each line and pass it to a routine. Thanks once again to all of you. |
Look at it this way, 'cut' is a one shot utility (Like eg wc). To process 1 million recs, you'd have to invoke it 1 million times.
'awk' is a programming lang and you should (I'm not an awk man, I'd use Perl) be able to invoke awk once(!) and write the entire (1 million recs) process inside that one awk process (I believe...). |
Quote:
Code:
cut -f1 -d"," file_million_records.txt |
Quote:
|
All times are GMT -5. The time now is 12:55 PM. |