LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-29-2008, 09:17 PM   #1
Jmor
LQ Newbie
 
Registered: Oct 2008
Location: Australia
Posts: 1

Rep: Reputation: 0
Merge many files in to one big file. like 20 file merge in one big file


Fore e.g.

File1.csv, Files.csv.......File20.csv. all the files are zip file and I required header only from first file remaining file header is not required bcz header is all the files are same. Please advice me how i will write script.

Fisrt unzip all the file
Merge all file into one file
But header required from first file.

Advice me
 
Old 10-29-2008, 10:11 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,411

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
1. gunzip *
2. cat file1 into new file
3. for each file (not inc file1), use wc -l to count lines file
4. cat fileN, pipe through tail using value of (wc -l -1) append to new file

man gzip
man wc
man tail
http://www.tldp.org/LDP/abs/html/

Last edited by chrism01; 10-30-2008 at 01:54 AM. Reason: remove duplicate line
 
Old 10-29-2008, 10:41 PM   #3
dv502
Member
 
Registered: Sep 2006
Location: USA - NYC
Distribution: Whatever icon you see!
Posts: 642

Rep: Reputation: 57
Code:
#!/bin/bash

# Extract all the zip files

for i in *.zip
do unzip "$i"
done

# Create an empty file called main.cvs
touch main.cvs

# Print the first line ONLY of file1.cvs to main.cvs
sed -n '1p' file1.cvs > main.cvs

# Append all cvs files minus the first line to main.cvs

for i in file*.cvs
do
sed -n '2,$p' "$i" >> main.cvs
done
To customize the script for your situation, replace the red with your actual file names.

- Cheers

Last edited by dv502; 10-29-2008 at 11:37 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
CSV file merge help lmedland Programming 2 06-04-2008 12:11 PM
Merge lines from a file lourencojunior Linux - Newbie 5 05-12-2008 02:54 PM
File merge frustration! the-yikes Slackware 2 04-28-2007 10:01 PM
awk command to merge columns from two separate files into single file? johnpaulodonnell Linux - Newbie 4 01-23-2007 11:10 AM
Is there a command to merge two files as two columns of one file? davee Linux - General 2 07-19-2005 11:52 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration