Merge many files in to one big file. like 20 file merge in one big file

Jmor · 10-29-2008, 08:17 PM

Fore e.g.

File1.csv, Files.csv.......File20.csv. all the files are zip file and I required header only from first file remaining file header is not required bcz header is all the files are same. Please advice me how i will write script.

Fisrt unzip all the file
Merge all file into one file
But header required from first file.

Advice me

chrism01 · 10-29-2008, 09:11 PM

1. gunzip *
2. cat file1 into new file
3. for each file (not inc file1), use wc -l to count lines file
4. cat fileN, pipe through tail using value of (wc -l -1) append to new file

man gzip
man wc
man tail
http://www.tldp.org/LDP/abs/html/

dv502 · 10-29-2008, 09:41 PM

Code:

#!/bin/bash

# Extract all the zip files

for i in *.zip
do unzip "$i"
done

# Create an empty file called main.cvs
touch main.cvs

# Print the first line ONLY of file1.cvs to main.cvs
sed -n '1p' file1.cvs > main.cvs

# Append all cvs files minus the first line to main.cvs

for i in file*.cvs
do
sed -n '2,$p' "$i" >> main.cvs
done

To customize the script for your situation, replace the red with your actual file names.

- Cheers