Latest LQ Deal: Latest LQ Deals
Go Back > Forums > Linux Forums > Linux - Server
User Name
Linux - Server This forum is for the discussion of Linux Software used in a server related context.


  Search this Thread
Old 01-17-2008, 11:53 PM   #1
Registered: Mar 2007
Distribution: Redhat &CentOS
Posts: 598

Rep: Reputation: 30
Split a gzip file

Hi Guys,

I have to download a log file , from my remote server to my desktop. The size of the file is around 35GB and it is a database dump file. I have "GZIP" it to around 5GB. Still it is showing the estimated time to download is around 39 hours.

I find , there is a split command. But using the man pages of split, i am unable to split it into pieces. For example, I want the fail to be break into pieces of 512MB. How can i do that?

Please Advise.
Thanks in Advance.
Old 01-18-2008, 12:46 AM   #2
Registered: Sep 2007
Location: Shetland, UK
Distribution: Slackware, Mandrake, LFS
Posts: 59

Rep: Reputation: 16
Try "split -b 500k filename"
Old 01-18-2008, 01:44 AM   #3
Registered: Mar 2007
Distribution: Redhat &CentOS
Posts: 598

Original Poster
Rep: Reputation: 30
Thank You SMbell100,

If i try with "split -b 500k Filename " , how could be the splited files appear?
Old 01-18-2008, 01:52 AM   #4
LQ Newbie
Registered: Jun 2007
Posts: 1

Rep: Reputation: 0
You can split the zipped file using the command:split -b 500k filename prefix,but unfortunately you can not unzip the small files independently unless you cat the all small files again.
Old 01-18-2008, 07:57 AM   #5
LQ Guru
Registered: Jan 2001
Posts: 24,149

Rep: Reputation: 267Reputation: 267Reputation: 267
I've had some bad experiences splitting up a compressed file. You might want to do the database dump and pipe it thru split by line numbers, then compress them for the copy.

When you split a database dump by size, you might cut it off right in the middle of a sql statement. Do it by line numbers, at least it will stay intact and then when you copy over the split dump files, there's really no need to put them back together, instead just make sure to reimport in the correct order or whatever you need to do with them.
Old 01-19-2008, 09:44 PM   #6
Registered: Mar 2007
Distribution: Redhat &CentOS
Posts: 598

Original Poster
Rep: Reputation: 30
Thank you all
Old 08-21-2012, 10:30 AM   #7
LQ Newbie
Registered: Jul 2012
Posts: 9
Blog Entries: 1

Rep: Reputation: Disabled
you can also split by the number of lines
this way you should specify that split is using the standard input by using the -
zcat dbpedia-data-0.nq.gz | split -l 2000000 - dbpedia0


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
gzip in new file salmanucit Linux - Software 6 06-30-2007 08:32 AM
How to split file , .. awk or split ERBRMN Linux - General 9 08-15-2006 12:02 AM
gzip a directory into a file tcma Linux - Software 4 10-07-2004 02:51 AM
Zip/Gzip "split" archives? Megamieuwsel Linux - General 2 01-10-2004 09:59 AM
How to gzip a file SnowSurfAir Linux - Software 5 07-31-2003 01:09 PM > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 10:35 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration