Shell script question
Hi Experts!!
I have backup_server and application_server. backup_server has directory AAA. I need to check from application server that is there any new files created today in the AAA dirctory. if yes, all files were created today or partial files?. I am not expert in scripting. Please suggest me the way. Thanks Dip |
I would recommend using the find command. Find has atime and mtime switches that will let you know which files were created or modified today.
Find docs: http://linux.die.net/man/1/find |
You can use two tests in you find statment to find files created in last 24 hours and files more than 5 minutes old which might filter out files still being written to. (Depending on the file size)
For currently open files, look at using the lsof command. To track files being created, look at the famd daemon. You will need to provide more information on what you mean by partial files, such as what kinds of files you are talking about (can integrity be tested ). How they were created, what log files if any will indicate a failure ( e.g. ftp logs ) |
Thanks for the reply.
Quote:
It says invalid command. I believe because its accessing remote server. Can someone modify the code to make it work. I also tried 'find' same way but it says usage: ls remote-directory local-file Thanks, dip |
Can I ask how files are considered as partial?
To find all the files in AAA directory: Code:
find "<path to AAA directory>" -type f -daystart Code:
find "<path to AAA directory>" -type f -daystart -not -ctime +0 Code:
find "<path to AAA directory>" -type f -daystart -ctime +0 Code:
find ... | wc -l Code:
#!/bin/bash Code:
#!/bin/bash |
I was already done with post I just made and was about to make the post when I read your new post in preview so please don't mind the scripts. I think they're not easy to apply if you're using ftp.
|
Quote:
Also, if you're going to use FTP, try either using an Expect script, or using SCP/SFTP with a key swap, so it won't prompt you for passwords. |
Isn't this all re-inventing the wheel?
dipeshvshah, Are you after actually knowing if the files were updated or are you more specifically just after updating all new and all updated files? if so I would recommend the rsync utility as this will do most of this on your behalf tho I do not know if it can be used to create a report as I myself never use rsync but I would presume it would be possible (as I back-up whole 30~50GB VM images as opposed to single files). |
You could use the find command locally to produce a list of filenames that meet your criteria. You don't use find inside an ftp script. In your ftp script you generated, you are including shell script commands. An ftp script can only include ftp commands.
You could run find on the remote server via ssh to return a list of files on the remote server matching certain criteria. One technique to consider is to use find's -printf command to generate the parts of the ftp script where you put the files. find /<dir>/ -type f -cmin +5 -ctime -1 -printf "put %p\n" >>sendfiles.ftp echo "bye" >>sendfiles.ftp Assemble the ftp script and then run it like ftp <sendfiles.ftp >logfile Be sure to read the info file for find. You can return the names of files and ignore directories, for example. ---- Using rsync or tar incremental dumps may be more suitable for what you want to do. You need ssh access to do this. |
Thanks jschiwal & konsolebox.
I used following approach to get listing of all files of remote server. Now I have remote server file information on same server. Quote:
Quote:
me a clue. Thanks |
Code:
ls "-lrt *.pdf" If not, I think it would be better (if you have access) to ssh into the remote server so you can run the find command. As already mentioned using sftp as well would be more secure. --- You can match the date "Aug 17" in this case, and count the number of occurrences using awk. or use sed or grep as before & wc to count the number of matches. TODAY=$( date '+%d %b' ) newfiles=$( sed -n "/$TODAY/p" list_of_files.txt | wc -l ) or newfiles=$( grep -c "$TODAY" list_of_files.txt) |
All times are GMT -5. The time now is 09:21 PM. |