Amanda [out of tape problem]! SOS!
 

I'm a newbie in using Amanda. Now I have a problem:

We use Amanda to backup our two servers: pv-chem-file and webserver, follows are from disklist file:
# Backup the file server
pv-chem-file / comp-high
pv-chem-file /home/pv-chem-file comp-high
pv-chem-file /usr comp-high

# Backup the web server
webserver / comp-high
webserver /home/webserver comp-high
webserver /usr comp-high

And we use cron to run amdump on every Sunday:
00 12 * * 7 /usr/sbin/amdump weekly

The problem is that sometimes it failed to dump /home/pv-chem-file and /home/webserver. For example, from 28 Aug to 4th Sept it failed on 2 consecutive Sundays, follows are from the log on 4th Sept:

DISK planner webserver /usr
START planner date 20050904
WARNING planner tapecycle (1) <= runspercycle (1)
WARNING planner Last full dump of pv-chem-file:/ on tape chemistry-weekly overwritten on this run.
WARNING planner Last full dump of pv-chem-file:/home/pv-chem-file on tape chemistry-weekly overwritten on this run.
WARNING planner Last full dump of pv-chem-file:/usr on tape chemistry-weekly overwritten on this run.
WARNING planner Last full dump of webserver:/ on tape chemistry-weekly overwritten on this run.
WARNING planner Last full dump of webserver:/home/webserver on tape chemistry-weekly overwritten on this run.
WARNING planner Last full dump of webserver:/usr on tape chemistry-weekly overwritten on this run.
FINISH planner date 20050904
STATS driver startup time 100.343
START taper datestamp 20050904 label chemistry-weekly tape 0
SUCCESS dumper pv-chem-file / 20050904 0 [sec 148.521 kb 149669 kps 1007.7 orig-kb 224090]
SUCCESS taper pv-chem-file / 20050904 0 [sec 59.055 kb 149670 kps 2534.4 {wr: writers 4679 rdwait 0.000 wrwait 56.207 filemark 2.734}]
SUCCESS dumper webserver / 20050904 0 [sec 369.658 kb 189277 kps 512.0 orig-kb 267935]
SUCCESS taper webserver / 20050904 0 [sec 73.718 kb 189278 kps 2567.6 {wr: writers 5916 rdwait 0.000 wrwait 70.963 filemark 2.612}]
SUCCESS dumper pv-chem-file /usr 20050904 0 [sec 422.849 kb 387148 kps 915.6 orig-kb 599555]
SUCCESS taper pv-chem-file /usr 20050904 0 [sec 423.688 kb 387149 kps 913.8 {wr: writers 12100 rdwait 254.809 wrwait 166.316 filemark 2.240}]
INFO taper tape chemistry-weekly kb 1025792 fm 4 writing file: Input/output error
FAIL taper webserver /usr 20050904 0 [out of tape]
ERROR taper no-tape [[writing file: Input/output error]]
FAIL dumper webserver /usr 20050904 0 ["data write: Connection reset by peer"]
sendbackup: start [webserver:/usr level 0]
sendbackup: info BACKUP=/sbin/dump
sendbackup: info RECOVER_CMD=/usr/bin/gzip -dc |/sbin/restore -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
| DUMP: Date of this level 0 dump: Sun Sep 4 12:18:43 2005
| DUMP: Dumping /dev/i2o/hda2 (/usr) to standard output
| DUMP: Label: /usr
| DUMP: Writing 10 Kilobyte records
| DUMP: mapping (Pass I) [regular files]
| DUMP: mapping (Pass II) [directories]
| DUMP: estimated 1827252 blocks.
| DUMP: Volume 1 started with block 1 at: Sun Sep 4 12:18:50 2005
| DUMP: dumping (Pass III) [directories]
| DUMP: dumping (Pass IV) [regular files]
| DUMP: 51.35% done at 3127 kB/s, finished in 0:04
FAIL driver webserver /usr 20050904 0 [dump to tape failed]
FAIL driver webserver /home/webserver 20050904 1 [no more holding disk space]
FAIL driver pv-chem-file /home/pv-chem-file 20050904 1 [no more holding disk space]
FINISH driver date 20050904 time 2537.978

From the above log, three disks webserver:/usr, webserver:/home/webserver, pv-chem-file:/home/pv-chem-file are failed to be dumped!

But in the next Sundays: 11th Sept, 9th Oct, 16th Oct, all the 6 disks are sucessfully dumped. Follows are part of the log on 16th Oct:

...
START taper datestamp 20051016 label chemistry-weekly tape 0
FINISH planner date 20051016
STATS driver startup time 113.450
SUCCESS dumper pv-chem-file / 20051016 0 [sec 175.465 kb 172296 kps 981.9 orig-kb 265740]
SUCCESS taper pv-chem-file / 20051016 0 [sec 67.482 kb 172297 kps 2553.2 {wr: writers 5386 rdwait 0.000 wrwait 64.646 filemark 2.703}]
SUCCESS dumper webserver / 20051016 0 [sec 361.068 kb 189467 kps 524.7 orig-kb 261145]
SUCCESS taper webserver / 20051016 0 [sec 73.740 kb 189468 kps 2569.4 {wr: writers 5922 rdwait 0.000 wrwait 71.008 filemark 2.587}]
SUCCESS dumper pv-chem-file /usr 20051016 0 [sec 430.299 kb 415071 kps 964.6 orig-kb 613820]
SUCCESS taper pv-chem-file /usr 20051016 0 [sec 431.321 kb 415072 kps 962.3 {wr: writers 12972 rdwait 255.756 wrwait 172.938 filemark 2.273}]
SUCCESS dumper webserver /usr 20051016 0 [sec 834.633 kb 418444 kps 501.4 orig-kb 940255]
SUCCESS taper webserver /usr 20051016 0 [sec 834.833 kb 418445 kps 501.2 {wr: writers 13078 rdwait 719.309 wrwait 112.990 filemark 2.331}]
SUCCESS dumper webserver /home/webserver 20051016 0 [sec 2071.867 kb 2434287 kps 1174.9 orig-kb 2695865]
SUCCESS taper webserver /home/webserver 20051016 0 [sec 2072.154 kb 2434288 kps 1174.8 {wr: writers 76073 rdwait 1015.448 wrwait 1052.507 filemark 2.710}]
SUCCESS dumper pv-chem-file /home/pv-chem-file 20051016 0 [sec 5848.180 kb 9820951 kps 1679.3 orig-kb 6662110]
SUCCESS taper pv-chem-file /home/pv-chem-file 20051016 0 [sec 5848.510 kb 9820952 kps 1679.2 {wr: writers 306906 rdwait 2077.862 wrwait 3757.541 filemark 2.401}]
INFO taper tape chemistry-weekly kb 13450784 fm 6 [OK]
FINISH driver date 20051016 time 9983.473

But in the other Sundays, 18th, 25th Sept, 2nd, 23th, 30th Oct., 6th, 13th, 20th, 27th, Nov., Amanda always failed to dump some of the disks, sometimes webserver:/usr, webserver:/home/webserver, /home/ pv-chem-file:/home/pv-chem-file failed, sometimes last two, sometimes only the last one. And the first failure is always caused by "out of tape", and then followed by "no more holding disk space" error.

Follows are holding disk config in the file amanda.conf:
holdingdisk hd1 {
comment "main holding disk"
directory "/usr/local/apps/amanda/dumps/" # where the holding disk is
use 500 Mb # how much space can we use on it
# a non-positive value means:
# use all space but that value
chunksize 1Gb # size of chunk if you want big dump to be
# dumped on multiple files on holding disks
# N Kb/Mb/Gb split images in chunks of size N
# The maximum value should be
# (MAX_FILE_SIZE - 1Mb)
# 0 same as INT_MAX bytes
}

I don't think it's the holding disk space problem because it cannot explain that why sometimes it succeeds, sometimes it doesn't.

Is it the tape problem? I suspect that it's not rewinded to its beginning. We have 5 tapes altogether and change it once every week.

Who can help me?! Thanks for any of ur suggestions!

Henry