LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 02-07-2007, 11:57 PM   #1
nazs
Member
 
Registered: Apr 2005
Posts: 57

Rep: Reputation: 15
How to remove first 2 lines of a file in a script


Hi All,
I have a file that is ftp'd each night. The file needs to have the first 2 lines deleted before being sent via ftp. How do I put that into the script? It needs to open the file,remove first two lines and then save the file with the lines removed.

Thanks,
Nazs
 
Old 02-08-2007, 12:28 AM   #2
nadroj
Senior Member
 
Registered: Jan 2005
Location: Canada
Distribution: ubuntu
Posts: 2,539

Rep: Reputation: 59
Code:
#!/bin/csh

set file = $1

cp $file $file.tmp

@ num = `cat $file | wc -l` - 2

tail -$num $file.tmp > $file

rm $file.tmp
just a quick and simple script with no error checking, etc.. but seems to do what you want. maybe instead of renaming the file to $file.tmp make it $file.tmpX, where X is some variable.. such as the date, and get rid of the rm command. so you always have a backup of each one

hope this helps.
 
Old 02-08-2007, 12:28 AM   #3
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
Code:
sed -n -i '3,$ p' filename
 
Old 02-08-2007, 12:29 AM   #4
nadroj
Senior Member
 
Registered: Jan 2005
Location: Canada
Distribution: ubuntu
Posts: 2,539

Rep: Reputation: 59
..or that!
 
Old 02-08-2007, 12:49 AM   #5
kshkid
Member
 
Registered: Dec 2005
Distribution: RHEL3, FC3
Posts: 383

Rep: Reputation: 30
Code:
awk '{ if( NR > 2 ) { print } }' currentfile > newfile
 
Old 02-08-2007, 01:02 AM   #6
nazs
Member
 
Registered: Apr 2005
Posts: 57

Original Poster
Rep: Reputation: 15
Thank you all for the quick responses. I still have some questions. Each file that is created has a counter. The file will always start with 2351 and then has a counter after that. The script looks for 2351*.OUT . So it might be 2351AA.OUT. The next one would be 2351AB.OUT and so on. For now i was trying the sed -n -i '3,$ p' filename which i put in bold and red. So i am not sure what to put down for the file name after sed -n -i '3,$ p' filename. Here is my script below.


#!/bin/sh

UNIDUMP=/usr/pa/universe/unidump/
ARCHIVE=/usr/pa/universe/unidump/bin_uni/archive/
LOG=/var/tmp/unisend.log

cd $UNIDUMP

#CHECK IF THERE ARE ANY FILES
gunzip $UNIDUMP/2351*.OUT.gz

if [ `find . -type f -maxdepth 1 -name "2351*.OUT*" | wc -l` -eq 0 ]; then
echo "Exit NO FILES"
exit
else

COUNTER=`find . -type f -maxdepth 1 -name "2351*.OUT"`

for i in $COUNTER;do
sed -n -i '3,$ p' filename

echo "record removed"

su slp -c "/usr/bin/gpg --no-tty -vsear AOB --passphrase-fd 0 $i< ~lsp/.gnupg/pw"


# FTP FILE TO AOB


Thanks again,
Nazs
 
Old 02-08-2007, 01:22 AM   #7
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
Please post code in [code] tags to improve readability.

No need to assign the list of files to a variable and then use it. Use xargs:

Code:
...
find . -type f -maxdepth 1 -name "2351*.OUT" | xargs sed -n -i '3,$ p'
...that assuming you wish to rmove the first two lines of all the files which find identifies.
 
Old 02-08-2007, 01:47 AM   #8
nazs
Member
 
Registered: Apr 2005
Posts: 57

Original Poster
Rep: Reputation: 15
I am getting this error:

sed: invalid option -- i
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...

-n, --quiet, --silent
suppress automatic printing of pattern space
-e script, --expression=script
add the script to the commands to be executed
-f script-file, --file=script-file
add the contents of script-file to the commands to be executed
--help display this help and exit
-V, --version output version information and exit

If no -e, --expression, -f, or --file option is given, then the first
non-option argument is taken as the sed script to interpret. All
remaining arguments are names of input files; if no input files are
specified, then the standard input is read.

E-mail bug reports to: bug-gnu-utils@gnu.org .
Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.

Nazs
 
Old 02-08-2007, 04:01 AM   #9
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
What do you get from this command:
Code:
sed --version
 
Old 02-08-2007, 04:38 AM   #10
kshkid
Member
 
Registered: Dec 2005
Distribution: RHEL3, FC3
Posts: 383

Rep: Reputation: 30
the sed version OP is using should probably be
Code:
GNU -- 3.02
 
Old 02-08-2007, 05:40 AM   #11
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
Wow, that's quite old. The version I have is 4.1.5. You can use a temporary file instead of the -i option (which says to edit a file in-place):
Code:
find . -type f -maxdepth 1 -name "2351*.OUT" | while read file; do
    sed -n '3,$ p' "$file" > "$file.new"
    mv "$file.new" "$file"
done
 
Old 02-08-2007, 07:29 AM   #12
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 719

Rep: Reputation: 72
Hi.

Here is an alternative to sed for skipping the first 2 lines of a file, namely by using tail:
Code:
#!/bin/sh

# @(#) s1       Demonstrate tail copying from beginning.

rm -f t1 t2
seq 100 >t1
L1=$( cat t1 | wc -l )
echo
echo " File t1 is $L1 lines long."

tail +3 t1 > t2
echo
L2=$( cat t2 | wc -l )
echo " File t2 is $L2 lines long."

echo
echo " Here are the first 10 lines of t2:"
head -10 t2
When run, this produces:
Code:
% ./s1

 File t1 is 100 lines long.

 File t2 is 98 lines long.

 Here are the first 10 lines of t2:
3
4
5
6
7
8
9
10
11
12
Best wishes ... cheers, makyo

( edit 1: typo )

Last edited by makyo; 02-08-2007 at 07:31 AM.
 
Old 02-08-2007, 11:49 AM   #13
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
You could also do it with ed. This command will work so long as your echo command supports the -e option (expand \ escaped characters). This is another nice one in that it doesn't create a second file - it modifies in-place:
Code:
echo -e "1,2 d\nw" | ed file
...or if your echo command doesn't, you might try it like this (which is a little more portable to some older and/or non-GNU systems):
Code:
(echo "1,2 d" ; echo "wq")| ed file
Again, you'd wrap this up in a loop as I described with the second sed examples (not the xargs one).
 
Old 02-08-2007, 11:07 PM   #14
nazs
Member
 
Registered: Apr 2005
Posts: 57

Original Poster
Rep: Reputation: 15
I just want to thank everyone for helping me. Mathewg42 this is what worked for me:

echo -e "1,2 d\nw" | ed file

Thanks alot!

Nazs
 
Old 02-18-2007, 11:34 PM   #15
cfaj
Member
 
Registered: Dec 2003
Location: Toronto, Canada
Distribution: Mint, Mandriva
Posts: 221

Rep: Reputation: 31
Quote:
Originally Posted by nazs
Thank you all for the quick responses. I still have some questions. Each file that is created has a counter. The file will always start with 2351 and then has a counter after that. The script looks for 2351*.OUT . So it might be 2351AA.OUT. The next one would be 2351AB.OUT and so on. For now i was trying the sed -n -i '3,$ p' filename which i put in bold and red. So i am not sure what to put down for the file name after sed -n -i '3,$ p' filename. Here is my script below.


#!/bin/sh

UNIDUMP=/usr/pa/universe/unidump/
ARCHIVE=/usr/pa/universe/unidump/bin_uni/archive/
LOG=/var/tmp/unisend.log

cd $UNIDUMP

#CHECK IF THERE ARE ANY FILES
gunzip $UNIDUMP/2351*.OUT.gz

if [ `find . -type f -maxdepth 1 -name "2351*.OUT*" | wc -l` -eq 0 ]; then
echo "Exit NO FILES"
exit
else

COUNTER=`find . -type f -maxdepth 1 -name "2351*.OUT"`

for i in $COUNTER;do
sed -n -i '3,$ p' filename

echo "record removed"

su slp -c "/usr/bin/gpg --no-tty -vsear AOB --passphrase-fd 0 $i< ~lsp/.gnupg/pw"
You don't need to check whether there are any files:

Code:
for i in 2351*.OUT
do
  {
    read; read; cat
  } < "$i" > TEMP
  mv TEMP "$i"
done
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Make a script remove lines from a file? spiffytech Linux - Software 5 12-29-2005 12:50 PM
How to remove lines from a file doza Linux - General 2 04-27-2005 12:59 PM
remove identical lines in a file benjithegreat98 Linux - General 4 04-24-2004 07:12 AM
How do i remove blank lines from a file? kakho Programming 1 04-15-2004 04:57 AM
[bash] remove lines from a file Drimo Programming 3 03-20-2004 12:16 PM


All times are GMT -5. The time now is 07:46 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration