.tar.gz 111GB extracting fails
 

Hi guys

We are trying to move Oracle applications database tier archive, that is 111GB (over Linux) to a USB external drive using cp. Though the file successfully gets transferred to the external drive, trying to extract the file from a 2nd machine always fails, saying the archive is corrupt.
We have checked the integrity of the archive using 7-zip, reporting no errors. However totally frustrated as our last few attempts were totally futile.

The interesting part is, if we do scp to transfer the file to 2nd machine, extraction doesn't fail.

Please let us know, how we can successfully move this archive to the 2nd machine which is at a remote location and no possibilities of setting up a FTP for such a huge size file.

Both the source and destination Linux distros are RHEL 5 Enterprise, 64Bit, ext3 file systems.


regards,

I guess there is a problem with the filesystem on that usb stick.

what format was the usb drive ?
the single file is after all over 100 Gig so the default from the manufacture fat32 is not being used

for RHEL 5.11 use ext3 on the usb drive

Check the size of usb drive? Is it greater than 111GB?

Run 'badblocks' command to find bad sectors in the usb drive.

There have been problems with 7zip corruptions, suggest you use something like bzip2, (or even tar & gzip).

Get the md5sum of the file.

After transfer, check that. If correct, then there's nothing wrong with the file.

Check versions of 7zip between the two machines.

Sorry for the delayed replies. Was lost in few meetings. I don't think the issues are related to file system. Both the external HDDs I used were formatted using ext3 (as it is the file system used with the source system)
Both the external HDDs are newly bought, checked thoroughly for errors.
Most importantly, I don't have the same issues with the application tier archive, that is just 25GBs in size.

regards,

---------- Post added 05-21-15 at 03:22 PM ----------

Hello John

The USB HDD is formatted using ext3 (As the source system)

regards,

My 1st HDD is 1TB and 2nd HDD is 2TB, hence size was never the problem.

regards,

---------- Post added 05-21-15 at 03:24 PM ----------

I am using tar to extract the files. I used to 7-zip to check the integrity of the archive over Windows 8.

regards,

The source archive was NOT at all corrupt as I was able to clone from the same archives in the source system. It only gets corrupt once after copied to the external HDD. The most interesting part is 7-Zip never complained about any errors, however failed to extract the files in the mid of extraction processes.

regards,

You're "saying" it gets corrupt. My point here is to prove or disprove those statements. Run md5sum on the file copied to that external HDD. If it matches, then the file is fine. If not, then I agree that there's something wrong.

And the other point was also to check the versions of whatever extraction tool you're using.

I still think there is an issue either with that filesystem (I do not mean in general with ext3, but that one on the usb stick). or the underlying hardware. You can try to copy another "big" file (for example a video) and check integrity.

Dear rtmistler

I am copying the file once again to the external HDD now, will post you the md5sum results soon.

regards,

Hello Pan64

This is how I built the external HDD

Connected the new HDD to Red Hat live system
Delete the NTFS partition using fdisk
Created a new partition
mkfs.ext3 to format

As I mentioned with my earlier posts, the files as big as 25GB are extracting without any issues. Please let me know whether I am doing something wrong with the partitioning or formatting part.

regards,

Hello rtmistler

I ran the md5sum against both source and destination files, and the md5sum for both files are the same. However, while going through the tar outputs, I could see that there were multiple errors raised like the quoted below

So it looks like, related to read/write errors? The archive at source is created by "oraprod" user belonging to a group "oinstall", at the destination I am trying to extract the archive as "root"

Other than the above, I can't see any other issues.

regards,

probably there are different users (user ids) configured and the user/group who created the tar does not exist on the target host therefore it has no right to create files/dirs.
Tar handles only ids, so you may have the the same username with different ids to make such conflicts.