tar - file changed as we read it
 

mmm... im getting errors when running this command -

cd /vol/cha-work/_ARCHIVE/to_be_archived/audio/robert_test/

tar -cf /vol/cha-archive/audio/after_louise_1606.tar after_louise_1606/

tar: after_louise_1606/renders_from/from_clipster/160922_after_louise_prores422_cont_51_clp_jd: file changed as we read it
tar: after_louise_1606/renders_from/from_clipster: file changed as we read it
tar: after_louise_1606/renders_from: file changed as we read it
tar: after_louise_1606/conform_materials/feature/offlines/161019: file changed as we read it

I wouldn't say it was an error. It looks like files were changes during tar, that was reported, and that's all. I think you you need to check if those file were really modified.

well thats what i thought, its just a warning not an error then it stopped creating the tar -

tar: after_louise_1606/renders_from/from_clipster/160922_after_louise_prores422_cont_51_clp_jd: file changed as we read it
tar: after_louise_1606/renders_from/from_clipster: file changed as we read it
tar: after_louise_1606/renders_from: file changed as we read it
tar: after_louise_1606/conform_materials/feature/offlines/161019: file changed as we read it
tar: after_louise_1606/conform_materials/feature/offlines/reel_5/160804/ No effects guide for SRC check : file changed as we read it
tar: after_louise_1606/conform_materials/feature/offlines/reel_5/160804: file changed as we read it
tar: after_louise_1606/conform_materials/feature/offlines/reel_5: file changed as we read it
tar: after_louise_1606/conform_materials/feature/offlines: file changed as we read it
tar: after_louise_1606/conform_materials/feature: file changed as we read it
tar: after_louise_1606/conform_materials: file changed as we read it
tar: after_louise_1606/audio/Continuous Mix: file changed as we read it
tar: after_louise_1606/audio: file changed as we read it
tar: after_louise_1606/renders_for/for_VT/160925_after_louise_prores422_cont_51_clp_iz: file changed as we read it
tar: after_louise_1606/renders_for/for_VT: file changed as we read it
tar: after_louise_1606/renders_for/for_online/161117_after_louise_reel_05_dnx36_rec709_f2l_bl_or/1920x1080: file changed as we read it
tar: after_louise_1606/renders_for/for_online/161117_after_louise_reel_05_dnx36_rec709_f2l_bl_or: file changed as we read it
tar: after_louise_1606/renders_for/for_online: file changed as we read it
tar: after_louise_1606/renders_for: file changed as we read it
tar: after_louise_1606: file changed as we read it
something went wrong with the tar, please do manually

how do you think tar can create a tarfile for you if the sources are continuously changing? What is the expected result?

looks like hasnt been touched -

drwxrwxrwx 12 root cha-work 307 Mar 9 12:34 .
drwxrwxrwx 3 root cha-work 35 Mar 10 09:21 ..
drwxrwxrwx 12 root cha-work 368 Mar 9 11:51 audio
drwxrwxrwx 4 root cha-work 62 Mar 9 11:53 blfx
drwxrwxrwx 4 root cha-work 102 Mar 9 11:04 conform_materials
drwxrwxrwx 4 root cha-work 100 Mar 9 10:34 DCI
-rwxrwxrwx 1 root cha-work 4.0K Mar 9 11:04 ._.DS_Store
-rwxrwxrwx 1 root cha-work 6.1K Mar 9 11:48 .DS_Store
drwxrwxrwx 7 root cha-work 185 Mar 9 11:53 gfx
drwxrwxrwx 5 root cha-work 104 Mar 9 11:52 media
drwxrwxrwx 10 root cha-work 285 Mar 9 12:34 renders_for
drwxrwxrwx 8 root cha-work 182 Mar 9 11:04 renders_from
drwxrwxrwx 2 root cha-work 0 Mar 9 11:53 _temp
drwxrwxrwx 5 root cha-work 168 Mar 9 12:35 _tests

sorry, but I do not understand. if it was the directory /vol/cha-work/_ARCHIVE/to_be_archived the times here are irrelevant.

sorry that ls-lah is for the after_louise_1606/ directory

still unclear, that output has no real meaning if we do not know anything else about that issue.

mmm...

looks like its working as i have created a 10gig file using the dd comand and its located in untitledfolder/

tar -cf /vol/cha-archive/audio/untitledfolder.tar untitledfolder/

and that works without any warnings about file changed as we read it

but i cant work it out because the folder after_louise_1606 i know 100% def it is not being written/read to at all so i dont know why its giving me these warnings and then failing on the tar

oh well

You have no background processes, perhaps anti-malware scanners, filesystem checkers or similar, that could be touching files?

If you want to investigate further, you could take a look at inotify.

what about if i use any of these option with the tar command

--ignore-failed-read

--ignore-command-error

--warning=no-file-changed might be more relevant. However, I imagine that it would be better, although possibly non-critical, to find out what's actually causing the problem.

yes exactly work out the real problem why it doesnt do it in the first place but you think if i use that option you provided it will not fail the tar command?

If those are the only warnings that are being generated then tar should work ok (unless, for example, tar decides that it has surpassed a warning limit or that some combination of events is deemed a critical failure).

Give it a go.

You think its still tarring up those individual files its producing the warning for or it just gives up altogether and fails the whole tar as its producing loads of warnings for individual files?

I don't know. Did you try running tar with the --warning=no-file-changed option?

no not yet sorry hydrurga i will try it monday when im back at work as the script is on my work pc, i will let you know def

i know what i will do to check if the tar is the same as the source directory i will do this -

tar -cf - after_louise_1606/ | md5sum

md5sum after_louise_1606.tar

and if there is no corruption they should be the same md5sum figures

still no joy im afraid even when i put in at the end of the tar command --warning=no-file-changed the tar still fails

strange thing is when i did tar -cf - after_louise_1606/ | md5sum, it said the exact same things ie the same warning i was getting before but it managed to produce a md5sum string

How about you make a copy of the directory, run the tar on the original directory, then compare the directory with the directory copy you made in order to see if any files or directories have been updated (contents or metadata) in the meantime. If tar says that some files are being changed, you need to find out which ones.

Depending on the results, you can then do the same thing again but without running tar, just pausing 5 minutes or so, to see if it is tar itself that is somehow making the changes.

I still do not know if you got the same md5sum or not.

i think i know what the problem is, its because im running the tar command over an nfs share and not on the local drive, this is why im getting read errors as when i run the tar command on my local disk (copy the data over to my local disk) it creates the tar fine

has any else encountered this problem?

I still can't see any problem, I don't know why can't you accept: the files were modified during the tar process. Probably they were accessed from another host, I have no idea. But you may have time related problems too.

mmm...

so as its a share its probably opened somewhere else, i need to see the NFS settings on the server