LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-16-2013, 08:15 AM   #1
Sneha_Sridhar
LQ Newbie
 
Registered: Feb 2013
Location: Chennai, India
Posts: 6

Rep: Reputation: Disabled
shell script to delete first line in a file


Hello Guys,

I have a shell script that deletes the first line of the input file.

FIRST_ROW=`head -1 $FILE_NAME`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE_NAME
fi

I am not able to understand what exactly the if condition is testing. Could someone help me..

Thanks
 
Old 02-16-2013, 08:25 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
[[ "$FIRST_ROW" =~ ^the ]]
This check if the first line starts (^) with the
 
1 members found this post helpful.
Old 02-16-2013, 08:32 AM   #3
Sneha_Sridhar
LQ Newbie
 
Registered: Feb 2013
Location: Chennai, India
Posts: 6

Original Poster
Rep: Reputation: Disabled
Hi Druuna,

Thank you Druuna. I have modified my script to delete the first line of the file and to delete all the lines with a particular pattern.

FIRST_ROW=`head -1 $FILE_NAME`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE_NAME
fi

sed '/ACTION/d' $FILE_NAME

This seems to work, but the problem is it writes the output twice. First it deletes the first line and prints the other lines, then it prints the input file with the first line removing only the lines with the pattern. I require only output. Could you tell me how I can modify this. Thanks..
 
Old 02-16-2013, 09:04 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by Sneha_Sridhar
I have modified my script to delete the first line of the file and to delete all the lines with a particular pattern.

FIRST_ROW=`head -1 $FILE_NAME`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE_NAME
fi

sed '/ACTION/d' $FILE_NAME
This seems to work, but the problem is it writes the output twice. First it deletes the first line and prints the other lines, then it prints the input file with the first line removing only the lines with the pattern. I require only output. Could you tell me how I can modify this.
A question first: Do you need the part that checks for ^the? You talk about removing the first line, but i haven't heard you about the first line having to start with the.

If you do not need the first line to start with the then all you want can be done with one command. Here's a sed example:
Code:
sed -e '1d' -e '/ACTION/d' infile
The 1d part deletes the first line and the /ACTION/d part deletes lines that contain the string ACTION.

If you do need the the check:
Code:
sed -e '1{/^the/d}' -e '/ACTION/d' infile
I've been using sed to show you how it can be done, but the same can also be done using others tools (awk comes to mind).

The reason you get the output twice is because you first use awk to print the file and then use sed to print the file (output will differ from each other).

This might come in handy:

Bash resources:
Sed/Awk resources:

Last edited by druuna; 02-16-2013 at 09:06 AM.
 
1 members found this post helpful.
Old 02-16-2013, 09:29 AM   #5
Sneha_Sridhar
LQ Newbie
 
Registered: Feb 2013
Location: Chennai, India
Posts: 6

Original Poster
Rep: Reputation: Disabled
Thanks again Druuna...

Yes I need to look for 'the' in the first line.. your reply was very helpful.. just another quick question..

If I use awk function to remove the lines with the word 'ACTION' will i get the output written once to the output file..

Something like this..

awk 'match($0,"ACTION") == 0 {print $0}' $FILE_NAME

Thanks,

Sneha
 
Old 02-16-2013, 09:51 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by Sneha_Sridhar View Post
If I use awk function to remove the lines with the word 'ACTION' will i get the output written once to the output file..

Something like this..

awk 'match($0,"ACTION") == 0 {print $0}' $FILE_NAME
The command shown will print the output once. You need to redirect to put it in a file:
Code:
awk 'match($0,"ACTION") == 0 {print $0}' infile > outfile
BTW: This would be a nicer way of doing the same:
Code:
awk '!/ACTION/ { print }' infile
 
1 members found this post helpful.
Old 02-16-2013, 09:59 AM   #7
Sneha_Sridhar
LQ Newbie
 
Registered: Feb 2013
Location: Chennai, India
Posts: 6

Original Poster
Rep: Reputation: Disabled
Thank you druuna..

I will try this out and get back to you on how it works.. Appreciate your help

Sneha
 
Old 02-17-2013, 10:03 AM   #8
Sneha_Sridhar
LQ Newbie
 
Registered: Feb 2013
Location: Chennai, India
Posts: 6

Original Poster
Rep: Reputation: Disabled
Hi Drunna,

I tried using awk function to delete the lines with the word 'ACTION'. It works fine but the problem of output getting written twice still exists.. i have tried with sed, the way you suggested and there is no issue there.. any idea why this issue...??

$ ./script.sh Input.txt
second line

the line above is blank
testing deletion of ACTION
the line below is blank

1234
5678
9
testing deletion of ACTION, and other pattern like SYNCWORD, SYNCNAME
the last line
the very first line is header
second line

the line above is blank
the line below is blank

1234
5678
9
the last line
================
INPUT FILE
================
the very first line is header
second line

the line above is blank
testing deletion of SYNCACTION
the line below is blank

1234
5678
9
testing deletion of ACTION, and other pattern like SYNCWORD, SYNCNAME
the last line
==============
SCRIPT
===========
$ cat script.sh
#!/bin/bash
FILE=$1
FIRST_ROW=`head -1 $FILE`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE
fi
awk '!/ACTION/ {print}' $FILE

Just cuious to know

Thanks for your help..

Sneha
 
Old 02-17-2013, 11:02 AM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by Sneha_Sridhar View Post
Hi Drunna,

I tried using awk function to delete the lines with the word 'ACTION'. It works fine but the problem of output getting written twice still exists.. i have tried with sed, the way you suggested and there is no issue there.. any idea why this issue...??
Code:
#!/bin/bash
FILE=$1
FIRST_ROW=`head -1 $FILE`
if [[ "$FIRST_ROW" =~ ^the ]];then
  awk 'NR > 1 {print}' $FILE
fi
awk '!/ACTION/ {print}' $FILE
You are still using 2 separate commands (the green and blue parts), each with its own input and output and both executed independently.

The sed command I showed (this one: sed -e '1{/^the/d}' -e '/ACTION/d' infile) does the actions done with the separate green and blue commands in one go.

You need to merge the above actions into one awk statement or, if for whatever reason you cannot do that with awk, use another way to do this. That is the main reason I came up with the sed statement. In this case it is the "better" tool to use (I'm not saying in cannot be done using awk, my experience pointed me to sed).
 
1 members found this post helpful.
Old 02-17-2013, 06:06 PM   #10
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.


You should start learning which tools are best for which job.
  • head/tail = grab whole lines from the start or end of a file.
  • grep = Extract lines, and sometimes parts of lines, based on patterns.
  • cut = Simple extraction of columns. Fast but limited in what it can process.
  • sed = Line and pattern based editing. The name is short for "stream editor", because it always processes the input from first line to last.
  • ed = A full, scriptable, command-line text editor, capable of fairly advanced editing. Usually better than sed for multi-line operations.
  • awk = A powerful, full-featured text processing language that can do the jobs of all the above and more, but optimized for field and column-based input.


There are other tools too, most of them included as part of the coreutils package. "info coreutils" will list them.

Finally, the shell can also do a lot of useful string manipulations.

For your current needs, I would recommend sed or ed myself. An example of sed has been shown, but here's ed:

Code:
printf '%s\n' '1d' 'g/ACTION/d' 'w' | ed -s input.txt
The main advantage here is that ed can write it's edits directly back to the original file. gnu sed also has an "-i" option that will save the changes back to the original file, but it's not portable across implementations, and it's actually just using a tempfile behind the scenes anyway.



Here are a few useful sed references:
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt
http://www.catonmat.net/series/sed-one-liners-explained

=====
How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)

=====
Here are a few useful awk references:
http://www.grymoire.com/Unix/Awk.html
http://www.gnu.org/software/gawk/man...ode/index.html
http://www.pement.org/awk/awk1line.txt
http://www.catonmat.net/series/awk-one-liners-explained
 
2 members found this post helpful.
Old 02-18-2013, 12:49 AM   #11
Sneha_Sridhar
LQ Newbie
 
Registered: Feb 2013
Location: Chennai, India
Posts: 6

Original Poster
Rep: Reputation: Disabled
Thanks a lot for all the help and info... and i will stick to the formatting style going forward..
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] delete a matched line from the file+shell script novicunix Programming 13 02-01-2013 09:37 AM
delete a line from file using shell script Abid Malik Programming 2 10-16-2010 03:01 AM
unix shell script:How to delete the first line in a file?? rche3252 Programming 6 03-03-2010 08:32 AM
How to delete a line from a text file with shell script programming Bassam General 1 01-28-2004 09:51 PM


All times are GMT -5. The time now is 04:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration