shell script to delete first line in a file
 

Hello Guys,

I have a shell script that deletes the first line of the input file.

FIRST_ROW=`head -1 $FILE_NAME`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE_NAME
fi

I am not able to understand what exactly the if condition is testing. Could someone help me..

Thanks

This check if the first line starts (^) with the

Hi Druuna,

Thank you Druuna. I have modified my script to delete the first line of the file and to delete all the lines with a particular pattern.

FIRST_ROW=`head -1 $FILE_NAME`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE_NAME
fi

sed '/ACTION/d' $FILE_NAME

This seems to work, but the problem is it writes the output twice. First it deletes the first line and prints the other lines, then it prints the input file with the first line removing only the lines with the pattern. I require only output. Could you tell me how I can modify this. Thanks..

A question first: Do you need the part that checks for ^the? You talk about removing the first line, but i haven't heard you about the first line having to start with the.

If you do not need the first line to start with the then all you want can be done with one command. Here's a sed example:The 1d part deletes the first line and the /ACTION/d part deletes lines that contain the string ACTION.

If you do need the the check:I've been using sed to show you how it can be done, but the same can also be done using others tools (awk comes to mind).

The reason you get the output twice is because you first use awk to print the file and then use sed to print the file (output will differ from each other).

This might come in handy:

Bash resources:

• Linux Shell environment
• Bash Frequently Asked Questions (BashFAQ)
• Bash Pitfalls
• BASH Programming - Introduction HOW-TO
• Bash Guide for Beginners
• Advanced Bash-Scripting Guide

Sed/Awk resources:

• GNU sed, a stream editor
• The GNU Awk User's Guide

Thanks again Druuna...

Yes I need to look for 'the' in the first line.. your reply was very helpful.. just another quick question..

If I use awk function to remove the lines with the word 'ACTION' will i get the output written once to the output file..

Something like this..

awk 'match($0,"ACTION") == 0 {print $0}' $FILE_NAME

Thanks,

Sneha

The command shown will print the output once. You need to redirect to put it in a file: BTW: This would be a nicer way of doing the same:

Thank you druuna..

I will try this out and get back to you on how it works.. Appreciate your help :)

Sneha

Hi Drunna,

I tried using awk function to delete the lines with the word 'ACTION'. It works fine but the problem of output getting written twice still exists.. :( i have tried with sed, the way you suggested and there is no issue there.. any idea why this issue...??

$ ./script.sh Input.txt
second line

the line above is blank
testing deletion of ACTION
the line below is blank

1234
5678
9
testing deletion of ACTION, and other pattern like SYNCWORD, SYNCNAME
the last line
the very first line is header
second line

the line above is blank
the line below is blank

1234
5678
9
the last line
================
INPUT FILE
================
the very first line is header
second line

the line above is blank
testing deletion of SYNCACTION
the line below is blank

1234
5678
9
testing deletion of ACTION, and other pattern like SYNCWORD, SYNCNAME
the last line
==============
SCRIPT
===========
$ cat script.sh
#!/bin/bash
FILE=$1
FIRST_ROW=`head -1 $FILE`
if [[ "$FIRST_ROW" =~ ^the ]];then
awk 'NR > 1 {print}' $FILE
fi
awk '!/ACTION/ {print}' $FILE

Just cuious to know ;)

Thanks for your help..

Sneha

You are still using 2 separate commands (the green and blue parts), each with its own input and output and both executed independently.

The sed command I showed (this one: sed -e '1{/^the/d}' -e '/ACTION/d' infile) does the actions done with the separate green and blue commands in one go.

You need to merge the above actions into one awk statement or, if for whatever reason you cannot do that with awk, use another way to do this. That is the main reason I came up with the sed statement. In this case it is the "better" tool to use (I'm not saying in cannot be done using awk, my experience pointed me to sed).

Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.


You should start learning which tools are best for which job.

• head/tail = grab whole lines from the start or end of a file.
• grep = Extract lines, and sometimes parts of lines, based on patterns.
• cut = Simple extraction of columns. Fast but limited in what it can process.
• sed = Line and pattern based editing. The name is short for "stream editor", because it always processes the input from first line to last.
• ed = A full, scriptable, command-line text editor, capable of fairly advanced editing. Usually better than sed for multi-line operations.
• awk = A powerful, full-featured text processing language that can do the jobs of all the above and more, but optimized for field and column-based input.

There are other tools too, most of them included as part of the coreutils package. "info coreutils" will list them.

Finally, the shell can also do a lot of useful string manipulations.

For your current needs, I would recommend sed or ed myself. An example of sed has been shown, but here's ed:

The main advantage here is that ed can write it's edits directly back to the original file. gnu sed also has an "-i" option that will save the changes back to the original file, but it's not portable across implementations, and it's actually just using a tempfile behind the scenes anyway.



Here are a few useful sed references:
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt
http://www.catonmat.net/series/sed-one-liners-explained

=====
How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)

=====
Here are a few useful awk references:
http://www.grymoire.com/Unix/Awk.html
http://www.gnu.org/software/gawk/man...ode/index.html
http://www.pement.org/awk/awk1line.txt
http://www.catonmat.net/series/awk-one-liners-explained

Thanks a lot for all the help and info... and i will stick to the formatting style going forward..