Split a gzip file

ZAMO · 01-17-2008, 11:53 PM

Hi Guys,

I have to download a log file , from my remote server to my desktop. The size of the file is around 35GB and it is a database dump file. I have "GZIP" it to around 5GB. Still it is showing the estimated time to download is around 39 hours.

I find , there is a split command. But using the man pages of split, i am unable to split it into pieces. For example, I want the fail to be break into pieces of 512MB. How can i do that?

Please Advise.
Thanks in Advance.

smbell100 · 01-18-2008, 12:46 AM

Try "split -b 500k filename"

ZAMO · 01-18-2008, 01:44 AM

Thank You SMbell100,

If i try with "split -b 500k Filename " , how could be the splited files appear?

hywl51 · 01-18-2008, 01:52 AM

You can split the zipped file using the command：split -b 500k filename prefix,but unfortunately you can not unzip the small files independently unless you cat the all small files again.

trickykid · 01-18-2008, 07:57 AM

I've had some bad experiences splitting up a compressed file. You might want to do the database dump and pipe it thru split by line numbers, then compress them for the copy.

When you split a database dump by size, you might cut it off right in the middle of a sql statement. Do it by line numbers, at least it will stay intact and then when you copy over the split dump files, there's really no need to put them back together, instead just make sure to reimport in the correct order or whatever you need to do with them.

ZAMO · 01-19-2008, 09:44 PM

Thank you all

sallyde · 08-21-2012, 10:30 AM

you can also split by the number of lines
this way you should specify that split is using the standard input by using the -
zcat dbpedia-data-0.nq.gz | split -l 2000000 - dbpedia0