A Journey With 'mksysb'
Posted 01-21-2011 at 10:04 PM by mufy
Tags mksysb looping
When the shit hits the fan, you take cover. Or, you cold stay right there, enjoy the spray and get new insights. Well, that's what I did - stood there and it was a learning experience.
It all began with an 'mksysb' backup failing on one of my development boxes. I'd start the backup, leave it for the day, and find the process stuck at one point when I come in the next day.
I was kind of blind. I didn't know whether the process was looping, progressing or had entered an infinite world. A couple of good people online were willing to share my trouble to see if it could be resolved. According to one, such a behavior was expected in the case of a large file.
The problem was I did not have anything tangile to look at as I did not know where to look. Then I learned that under /tmp directory the 'mksysb' command created a directory as 'mksys.<PID>'. In it there was a file which got populated with the names of files that were getting processed. This was the breakthrough I needed. Identifying the file that was causing the issue and to take the troubleshooting to another level was what I intended to do.
It turns out an application process was hanging onto this particular file preventing 'mksysb' from progressing. Bringing down the application resolved that.
Sounds simple right? Even that's what I felt at the end of it all. The details of the discussion may be found here.
It all began with an 'mksysb' backup failing on one of my development boxes. I'd start the backup, leave it for the day, and find the process stuck at one point when I come in the next day.
Code:
Creating information file (/image.data) for rootvg.. Creating list of files to back up................. Backing up 238748 files.............................. 9251 of 238748 files (3%).............................. 19979 of 238748 files (8%).............................. 49828 of 238748 files (20%).............................. 56914 of 238748 files (23%).............................. 82208 of 238748 files (34%).............................. 82208 of 238748 files (34%).............................. 82208 of 238748 files (34%)..............................
The problem was I did not have anything tangile to look at as I did not know where to look. Then I learned that under /tmp directory the 'mksysb' command created a directory as 'mksys.<PID>'. In it there was a file which got populated with the names of files that were getting processed. This was the breakthrough I needed. Identifying the file that was causing the issue and to take the troubleshooting to another level was what I intended to do.
It turns out an application process was hanging onto this particular file preventing 'mksysb' from progressing. Bringing down the application resolved that.
Sounds simple right? Even that's what I felt at the end of it all. The details of the discussion may be found here.
Total Comments 0
