Bash: How to add a line on top of text files

guest · 02-05-2009, 02:36 PM

I have a bunch of CSVs (250 of them) that doesn't have a header, so I need to add a header

Does anyone know a command that will add a header to all files like so?
NAME, AGE, CITY, STATE

Thanks!

colucix · 02-05-2009, 02:39 PM

Using sed?

Code:

sed -i '1iNAME, AGE, CITY, STATE' file.csv

include this command in a for loop if you want to repeat for every CSV file.

guest · 02-05-2009, 02:49 PM

Is there a command that just edits all of them? like *.csv?

pixellany · 02-05-2009, 03:03 PM

Quote:

Originally Posted by guest

Is there a command that just edits all of them? like *.csv?

That works, but I have heard people argue that the loop is more robust. Why not try it??

So--2 choices:

sed -i '1iSTUFF' *.csv

for fil in *.csv; do sed -i 'i1STUFF' $fil; done

theNbomr · 02-05-2009, 03:35 PM

No need for sed here, bash has everything you need.

Code:

echo "NAME, AGE, CITY, STATE" > headerfile
for csv in *.csv; do cat headerfile $csv > tmpfile2; mv tmpfile2 $csv; done
rm headerfile

--- rod.

colucix · 02-05-2009, 04:43 PM

Quote:

Originally Posted by theNbomr

No need for sed here, bash has everything you need.

Code:

echo "NAME, AGE, CITY, STATE" > headerfile
for csv in *.csv; do cat headerfile $csv > tmpfile2; mv tmpfile2 $csv; done
rm headerfile

--- rod.

Not really. sed -i is one external command, while cat + mv + rm are three!

theNbomr · 02-05-2009, 05:14 PM

Quote:

Originally Posted by colucix

Not really. sed -i is one external command, while cat + mv + rm are three!

Doh! Yes, quite true. I guess I always think of these tiny tools as part of bash, and sometimes, they are as builtins. Good catch.
--- rod.

jan61 · 02-05-2009, 05:39 PM

Moin,

Quote:

Originally Posted by colucix

Not really. sed -i is one external command, while cat + mv + rm are three!

I wonder about the method sed uses to do the inplace editing. I think that it can't be done in-memory, some temporary files should have to be created. That's why I'd prefer this method (not tested):

Code:

mkdir new_dir
cd old_dir
for f in *.csv; do
  (echo "NAME, AGE, CITY, STATE"; cat $f) >../new_dir/$f
done
cd ..
rm -r old_dir
mv new_dir old_dir

This method uses only one simple cat instead of the sed monster and additionally allows you to break every time you receive an error and keep the original files as long as all new files are created, but temporary needs a free space on harddisk similar to the source folder's space.

Jan

theNbomr · 02-05-2009, 05:59 PM

Nice, jan61. Your method removes the need for one 'rm'.

Code:

for csv in *.csv; do (echo "NAME, AGE, CITY, STATE"; cat $csv) > tmp; mv tmp $csv; done

I knew someone would come up with a method for that. Thanks.
--- rod.

guest · 02-05-2009, 11:20 PM

Wow thanks guys!

You guys are such a life saver!!

guest · 02-06-2009, 12:29 AM

but wait.. what happens if there's spaces in the files? how do you handle that? the script doesn't work with filenames with spaces..

guest · 02-06-2009, 12:30 AM

nevermind.. this fixed it all! lol

sed -i '1iSTUFF' *.csv

'nuf sed

shyamkumar1986 · 02-07-2009, 06:23 AM

Quote:

Originally Posted by theNbomr

No need for sed here, bash has everything you need.

Code:

echo "NAME, AGE, CITY, STATE" > headerfile
for csv in *.csv; do cat headerfile $csv > tmpfile2; mv tmpfile2 $csv; done
rm headerfile

--- rod.

@theNbomr - Pretty neat script ... I was able to use this for another similar requirement I was working on today.

Thanks!!!