[SOLVED] How do I prepend a header into an already existing file?

smturner1 · 03-31-2014, 01:24 PM

I rebuilt a bunch of files (65 or so) that someone deleted for a specific customer, subaccount. All of the records are in the files with their related file name. Now I have to prepend the header and append the trailer to each one of these files.

As you all are aware that '>>' does not prepend files. I am lost because I know that awk and/or sed probably has the solution I am looking for but it is evading me.

I would appreciate any help with the following shell.

Code:

 cat <file_list> | while read file_name; do
 zcat $file_name | head -1 | while read hdr; do
 
     echo $hdr | awk {something goes here, copy to $file_name} 
 done
zcat  $file_name | tail -1 >> $file_name

Thank you for your help in advance.

suicidaleggroll · 03-31-2014, 01:48 PM

Why not just put it into a new file?

head -1 $file_name > new_file
cat existing_file >> new_file
tail -1 $file_name >> new_file

smallpond · 03-31-2014, 01:48 PM

Seems like for each file you want to create a header and trailer file, then in your inner loop do:

Code:

cat $header $file_name $trailer > $tmpfile ; mv -f $tmpfile $file_name

It's not clear to me from your post where you are getting your header and trailer from.

smturner1 · 03-31-2014, 02:04 PM

Suicidaleggroll - I could do that if all I was looking for is a quick and tedious solution.

smallpond - Let me see if I can clear this us a little.

Code:

cat <file_list> | while read file_name; do
 zcat $file_name | head -1 | while read hdr; do
 
     echo $hdr | awk {something goes here, copy to $file_name} 
 done
zcat  $file_name | tail -1 >> $file_name

file_list = a list of file names in a given directory (basically an array).
file_name = one of the files in "file_list" (basically a scalar). All of the files that are read to this var are compressed.

I hope this clears things up.

Two things to note here; 1 - I am not looking for a solution that "just" works (that's easy). 2 - I want clean efficient code that is reusable.

smturner1 · 03-31-2014, 05:01 PM

I guess I will concede, since there are no other options. Thank you both for your input....

Here is the solution I used (which is basically the same as the two contributors above).

Code:

cat file.txt | while read file_name; do
   cat $file_name | head -1 | while read hdr; do
    
      echo $hdr > temp_file
      cat $file_name | grep -i foo | grep T1234 >> temp_file
      cat $file_name | tail -1 >> temp_file
      cat temp_file > $file_name
      done
  
   rm temp_file
   done

PS - I am still looking for someone who may have a more elegant approach.

smallpond · 04-01-2014, 10:21 AM

In that case, here is a slightly more elegant solution. NR == 1 selects first line, tail holds last line at END, regexes select your body lines. One gotcha is that your trailer line must not match a regex or it will print twice.

Code:

cat file.txt | while read file_name; do
    awk '{tail=$0} NR == 1 || /foo/ || /T1234/ {print $0} END {print tail}' $file_name > temp_file
    mv -f temp_file $file_name
done

suicidaleggroll · 04-01-2014, 10:43 AM

sed can also prepend in-place with the -i and 1i flags (see below). However, internally it still does the same write-to-temp-file-and-move-back-into-place operation that the above steps do.

Code:

sed -i '1i Line to prepend' file.txt

And you should really get rid of that unnecessary second loop and all of those useless cats, they just complicate the code and slow things down with absolutely no benefit:

Code:

while read file_name; do
   head -1 $file_name > temp_file
   grep -i foo $file_name | grep T1234 >> temp_file
   tail -1 $file_name >> temp_file
   mv temp_file $file_name
done < file.txt