LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 07-20-2013, 02:55 PM   #1
DaemonHunter
LQ Newbie
 
Registered: Mar 2013
Posts: 13

Rep: Reputation: Disabled
STAT issues in bash script; please help!


I have been writing a bash script to sync files between my devices and a cifs mount hosted from a router. The issue I am having comes at the end of my script when the program copies files to either locations based on a greater modification time. Here is the last part of the script where the error occurs.
(find files is a function defined earlier in the file)
Quote:
findfiles > /tmp/findresults.txt

while read files;
do
if [[ "$stathome" -gt "$statnetwork" ]] && [[ "$stathome" -ne "$statnetwork" ]] && [[ ! -d "$hBasename"/"$files" ]]; then

sudo cp --preserve=timestamps "$hBasename"/"$files" "$nBasename"/"$files" 2>&1 > /dev/null && echo "Copied "$files" to network Teir1."
elif [[ "$statnetwork" -gt "$stathome" ]] && [[ "$stathome" -ne "$statnetwork" ]] && [[! -d "$hBasename"/"$files" ]]; then
sudo cp --preserve=timestamps "$nBasename"/"$files" "$hBasename"/"$files" 2>&1 > /dev/null && echo "Copied "$files" to home Teir1."
fi

stathome=$(stat "$hBasename"/"$files" | grep Modify* | cut --complement -c 1,2,3,4,5,6,7,8 | cut --complement -c 5,8,11,14,17,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35)
statnetwork=$(stat "$nBasename"/"$files" | grep Modify* | cut --complement -c 1,2,3,4,5,6,7,8 | cut --complement -c 5,8,11,14,17,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35)

done < /tmp/findresults.txt

Quote

The issue is that when I run "stat "~/.Teir1/base | grep Modify* | cut --complement -c 1,2,3,4,5,6,7,8 | cut --complement -c 5,8,11,14,17,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35" and the same command on /mnt/.Teir1/base, stat returns the proper Mod time for both locations which is 20130712141506, that is the cut down Modification time in both locations however when I run my program the file "base" in the home directory returns 20130610235652 and 20130610195652 in the network location. No matter how many times I run the program with cp --preserve=timestamps, the same values are indicated for the file in its respective location. I use a test version of the script that echo's the values returned for all files subjected to the stathome and statnetwork subshell functions. the base file used here is simply one example of a large number of files that return greater cut down mod times that exist in my home location and results in the program always copying these files. essentially after the program runs once the mod times should be the same in both locations.

So....why is stat returning the correct (and subsequently same times) for both the home and network files when run directly from the shell but different mod times when the program runs?

Any help is appreciated and I am aware that there are a handful of programs that do what my program does however I want to fix my program as an exercise in programming logic. Thanks to all who reply!
 
Old 07-20-2013, 07:15 PM   #2
linosaurusroot
Member
 
Registered: Oct 2012
Distribution: OpenSuSE,RHEL,Fedora,OpenBSD
Posts: 982
Blog Entries: 2

Rep: Reputation: 244Reputation: 244Reputation: 244
If something is complicated in bash that's a clue you should be using Perl - where stat() is a function and $mtime is one of the numeric values returned from it.
 
Old 07-20-2013, 11:13 PM   #3
DaemonHunter
LQ Newbie
 
Registered: Mar 2013
Posts: 13

Original Poster
Rep: Reputation: Disabled
Thanks for the heads up on the Pearl environment variables however I am trying to see if I can solve this issue in bash as the script is already written in bash and I am not versed enough in pearl.
 
Old 07-21-2013, 12:01 AM   #4
DaemonHunter
LQ Newbie
 
Registered: Mar 2013
Posts: 13

Original Poster
Rep: Reputation: Disabled
OK so I totally figured it out, the stat subshell functions needed to be defined directly after the if statements because what was happening was whe while loop was skipping the first line of the txt file and miss matching the mtimes becasue by the time the loop had defined "$stathome" and "$statnetwork" the index had changed

Last edited by DaemonHunter; 07-21-2013 at 12:05 AM.
 
Old 07-21-2013, 09:31 AM   #5
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,780

Rep: Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081
You can use a simpler method to get the modification time time:
Code:
stat -c %y "$nBasename/$files" | cut -d. -f1 | tr -dc 0-9
 
Old 07-21-2013, 10:47 AM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques. Thanks.

I'm not going to address the original question, since you seem to have figured that out for yourself, but I'd like to help you with some general scripting advice.

Using just the posted part in the OP, here's an example of how I would write it:

Code:
cp_files(){ sudo cp --preserve=timestamps "$@" ;}

while read files; do

    if [[ ! -d "$hBasename"/"$files" || "$stathome" -ne "$statnetwork" ]]; then

        continue

    fi

    if (( "$stathome" > "$statnetwork" )); then

        cp_files "$hBasename/$files" "$nBasename/$files" 2>&1 > /dev/null
        (( $? )) || echo "Copied "$files" to network Teir1."

    elif (( "$statnetwork" > "$stathome"  )); then

        cp_files "$nBasename/$files" "$hBasename/$files" 2>&1 > /dev/null
        (( $? )) || echo "Copied "$files" to home Teir1."

    fi

    stathome=$( stat -c %y "$hBasename"/"$files"g )
    stathome=${stathome%.*}
    stathome=${stathome//[ -:]}

    statnetwork=$( stat -c %y "$nBasename"/"$files"g )
    statnetwork=${statnetwork%.*}
    statnetwork=${statnetwork//[-: ]}

done < <( findfiles )
Notes:
1) I'm assuming that there's no need for the file except for processing here, and reconfigured it so that the loop reads directly from the findfiles function.

2) Testing for multiple conditions at once is generally sloppy and prone to error. I believe it's much cleaner to separate them out whenever possible. I'm guessing that the action you want to do here is skip over the rest of the commands and continue to the next file, but you can of course change it to something else.

3) When using advanced shells like bash or ksh, it's recommended to use [[..]] for string/file tests, and ((..)) for numerical tests. Avoid using the old [..] test unless you specifically need POSIX-style portability.

http://mywiki.wooledge.org/BashFAQ/031
http://mywiki.wooledge.org/ArithmeticExpression

One benefit of these is that, when you do need to test for multiple conditions, you can use && and || directly inside of them, instead of stringing together multiple brackets.

4) Functions can be used to pre-define commands with common options, reducing clutter.

5) moved the "success" messages into separate commands after the cp actions just to keep the code shorter and cleaner.

6) When quoting lines like "$hBasename/$files", the general rule of thumb is to simply quote the longest string possible. There's no need to quote each variable separately here; that's only necessary if the name can be conflated with the one following it (i.e. the next character is also [a-zA-Z0-9_]), and even then you can use the full ${var} form of the expansion instead to protect it.

7) I've further cleaned up ntubski's suggested modification to use bash parameter substitutions, which should make it even more efficient, although it does involve more lines of code.

However, there appears to be a big logic error here as well: the stathome and statnetwork variables will be overwritten on each iteration. Only the final file processed will be recorded at the end of the loop. If this is what you want, great, otherwise you may want to use an array or something instead.

Last edited by David the H.; 07-21-2013 at 10:51 AM. Reason: minor corrections
 
Old 07-21-2013, 11:42 AM   #7
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,780

Rep: Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081Reputation: 2081
Quote:
Originally Posted by DaemonHunter View Post
OK so I totally figured it out, the stat subshell functions needed to be defined directly after the if statements because what was happening was whe while loop was skipping the first line of the txt file and miss matching the mtimes becasue by the time the loop had defined "$stathome" and "$statnetwork" the index had changed
I think the stat calls should go before the if statements, not "directly after". Otherwise you are basing your decisions about copying a file on the timestamp of the previous file which doesn't really make sense...



@David the H.: You forgot to reverse the condition for the continue.
 
Old 07-21-2013, 01:17 PM   #8
DaemonHunter
LQ Newbie
 
Registered: Mar 2013
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
I think the stat calls should go before the if statements, not "directly after". Otherwise you are basing your decisions about copying a file on the timestamp of the previous file which doesn't really make sense...
You are right and thats actually how I implemented it in the script, I just miss spoke. As for the suggestions from David the H. I really appreciate the suggestions and like the cleaner style of your format. I will incorporate a decent amount of that into my script. Again I thank all who replied and would also like to add that I am relatively new to bash but am always on the look out for new ways of doing things!
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Bash script issues -- halp! danowar Programming 14 02-09-2012 03:37 PM
[SOLVED] Issues doing simple math in Bash script buee Linux - Newbie 10 07-20-2010 10:46 PM
bash script using sed/scp/ssh has issues with delimited file ScottThornley Programming 5 03-18-2009 03:45 PM
issues programming a script in BASH gravesb Linux - Software 1 07-07-2005 01:03 AM
stat function in BASH... PokerFace Programming 1 11-04-2002 11:31 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 12:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration