I posted it sometime back but couldn't get any help. May be somebody can help me now.

Here is the problem:

My ftp server is giving lot of trouble to all our clients since last 2 months but in a very inconsistent way. They get "CRC" error trying to uncompress the compressed files. The clients get exactly same size of the compressed file but they just can't uncompress it.

Here is what we have tried:

[1] We made sure that our clients issue "binary" command before they pull over the files,

[2] We tried with ftp clients like "ncftp", "wsftp" and "our good old normal ftp" but no success,

[3] We also tried all kind of compressions i.e. "gzip", '"compress", "zip", "bzip2" but no success.

[4] We tested and figured out that "uuencode" seems to work for these binary files but since it also increases the file size, our clients really hate that.

[5] Looks like the trouble happens with only big files (50Mb or more after compression). We tried with smaller compressed test cases but they all work (binary or ascii). That's what I call it inconsistency,

[6] This is strange: Gzipped files ftpd over to a linux box, give trouble on "gunzip" but when moved over to a windows machine, can successfully be uncompressed using "winzip". Since not all of my clients have windows machine, I can't ask them do the same..

[7] I know I could try with sftp/scp but the office legal system is not ready to give login access to outside clients. Hence that is possibly ruled out..

I was told that at times, this happens because of the extra checks that the ISP can put over the data. I am not sure what could be those to ask our ISP. They straight away put blame on us that we are doing something wrong. It was all working ok for 1 year. We are very sure, we haven't changed anything in our office setup in last 1 year.

Please help and plenty of thanks..

this sounds quite quirky and possibly network related. The first thing I would do is start checking checksums or file hashes. use md5sum to generate a (unique) checksum on your file, then transfer. when you get the error, use md5sum on the other end and compare. if the checksums are the same, you're files are binary (i.e. bit-by-bit) identical.

maybe there is data being lost and this is being padded out with null/zero/random data?

that's what I'd be checking, since it eliminates the network from the equation.

HTH

ps - the md5 checksum won't give you truly (i.e. mathematically) unique responses. I think you could expect a collision (i.e. different inputs giving same output) once every billion years or something like that. it's small enough not to worry about, but not identically zero.

We did "md5 checksum" and it gives identical results. This, I actually expected because of the point [6] that I have raised in my posting.

It is very inconsistent since the same file to same client can sometime be received ok and at times not although everytime the file byte size is exactly same at his end!

We have also noticed quite a delays in login prompt to appear while doing ftp (This I have posted in separate thread). I am not sure if two are related. Additionally our clients tell me that they get intermittant hickups (stalls) after every 10-15 minutes for 20-30 seconds when they are connected to our ftp site. So, this is sure that the problem does not arise with 10-20Mb file. But we deal with like 400Mb files