need help with text manipulation
Hello all I'm a newbie to programming and scripting but so far I been working with a friend on a bash script to automate the download/decryption/text-editing/upload of certain files.
Basically our script downloads a file from a predetermined FTP site, the file is encrypted using PGP so the script also decrypts the file and names it. The script has a lot of code that enables it to send an e-mail wanning to me if it encounters an error (File not been in the ftp site, name of file not matching or file been corrupted etc etc). Anyways its been a fun project so far but now i have hit a wall. My next step in the project is to do some basic text editing on the file basically i need to delete the first and last line of the file. the file looks something like this: 01030e939-3-20339302039393039403942-12-03920399203 jdkfjei349893894838294 98320498023948928490483489 3924803928409238409388 92304803928402384098320498 9203893r023jf039840390894403984 293-0482390840923840983209 32094802398409238409823039 39480293840923840983204980 3204832948023984023809483029480 932482039840239840392849048 230482309-02398402398402398409238 8320p498230948230948230984230984 4954868 Everyday i will be downloading a file that will have more or less data. but everytimne i have to delete the first and last line, i have read and tried AWK with no luck anybody can point me in the right direction please. |
sed
I think you can do such thin with the sed instruction
|
Thanks much for pointing out SED i found some stuff on line that looks like is going to help with this:
# delete the first 10 lines of a file sed '1,10d' so is this right ? #Delete the first line of a file sed '1d' also to delete last line sed '$d' Thanks much. |
For some reason when I try to combine sed '1d' and sed '$d' They dont play nice =(
|
Well, this suggestion isn't terribly elegant, but if you are using bash, you can use
wc -l to find how many lines in the file, then subtract 1 to get newlinecnt and use head, tail to get all but last line & first lines eg if t.t has 9 lines, then head -8 t.t|tail -7 chops off 1st & last lines. |
Quote:
Code:
sed '1d$d' Code:
sed -e '1d' -e '$d' |
You can either specify multiple commands using multiple -e arguments, or separate commands with a ;
Code:
matthew@chubby:~/tmp$ cat testfile |
Python alternative:
Code:
data = open("file").readlines() |
Quote:
|
Quote:
correct me if i am wrong, but even if sed, awk, grep over 10G file, its slow too right? |
Quote:
However, I think the approach above with python will try slurp the whole file into memory. |
Quote:
Quote:
|
Quote:
Yes that was the mistake i was doing thanks much matthewg42, thank you for the example it really helped. Thank you all very much for taking the time to help me out. |
All times are GMT -5. The time now is 06:06 PM. |