[SOLVED] How to select newest "dated" directory?

epipko · 03-13-2015, 05:17 PM

Hi,
I have a /backup directory which contains 5 days worth of backups:
2015-03-12_03-01-07
2015-03-11_03-01-07
2015-03-10_03-01-07
2015-03-09_03-01-07
2015-03-13_03-01-07

I need to copy the content of the NEWEST backup. Ideally, I'd like to find newest backup and assign it to a variable for later use.
I know I can

Code:

ls -lt

to sort directories, but how do I pick the first one and store it into a variable?

Thanks,

Kustom42 · 03-13-2015, 06:03 PM

There are several ways to do this. You could split the results out in perl or awk and do some crazy sorting. But from your structure it looks like you always just want to find the folder that was backed up most recently which will always be the one that was modified the same day.

Couple of ways I could do it.

Easily with ls and some piping, could have some reliability issues based upon when it is ran and whether there are other folders or files that could be in there that would have modified data within the past 24 hours:

Code:

#!/bin/bash
backupdir=$(ls -lt | grep -v ^total | head -n1 | awk '{print $9}')
echo $backupdir

Also if you always just want to find yesterdays backup:

Code:

#!/bin/bash
yesterday=$(date --date='1 day ago' +Y-%m-%d)
echo $yesterday
backupdir=$(find /backup -type d -name $yesterday*)
echo $backupdir

Again neither of these are very advanced but based on your OP I thought it would be best to keep it simple so that you can understand what you are doing and decipher it instead of spending 20 minutes on a perl/awk script that would just be confusing and overkill.

Let us know if you have any other questions and good luck

epipko · 03-14-2015, 06:32 PM

Thank you very much for quick answer. I don't know (yet) what the parts of the expressions are, but will try to break it apart to learn.

joe_2000 · 03-15-2015, 04:46 PM

You can get away with a little less piping when using

Code:

ls -1rt

Mind the first option flag is the number 1, not the letter L. (That way you only get names, rather than the whole other stuff that the -l flag produces).
The -rt part will inverse the order by age. If you pipe that into tail you can reduce the output to the newest directory:

Code:

ls -1rt | tail -n 1

Lastly, as already suggested, you can put the output into a variable like so:

Code:

newest_dir=$(ls -1rt | tail -n 1)

All of this obviously assumes you are in the right directory. If you run these commands in a script make sure to have the right cd command before them.

EDIT: Closer to Kustom42's suggestion you could also remove the -r flag and use head instead of tail. The advantage of the -r / tail combi is that when you run them manually you get to see the newest stuff at the bottom which avoids having to scroll back. For scripts obviously there is not difference between the two.

unSpawn · 03-15-2015, 06:21 PM

http://mywiki.wooledge.org/ParsingLs ergo +1 for using "date --date='1 day ago'".

epipko · 03-16-2015, 01:40 PM

Thanks again for your help.
One more question regarding the same issue:
Since I need to copy the content of that directory, how to store the result of the remote server search to a local variable so I can scp the content of it?

This does not work:

Code:

newest_dir=$(ssh root@rem_server ls -1rt /path/to/dir | tail -n 1)
scp -r root@rem_server:/path/to/$newestdir

Thank you

joe_2000 · 03-16-2015, 03:46 PM

Quote:

Originally Posted by epipko

Thanks again for your help.
One more question regarding the same issue:
Since I need to copy the content of that directory, how to store the result of the remote server search to a local variable so I can scp the content of it?

This does not work:

Code:

newest_dir=$(ssh root@rem_server ls -1rt /path/to/dir | tail -n 1)
scp -r root@rem_server:/path/to/$newestdir

Thank you

The first part works for me. You can test it by running only the first line and then outputting the variable.

Code:

echo $newest_dir

The second line looks wrong. You are providing only one command to your scp command, which does not make any sense.

epipko · 03-16-2015, 04:16 PM

I made a mistake pasting second line. It should read:

Code:

newest_dir=$(ssh root@rem_server ls -1rt /path/to/dir | tail -n 1)
scp -r root@rem_server:/path/to/$newest_dir /path/to/local_dir

You said that first line works for you, but it does not return anything back for me.

joe_2000 · 03-16-2015, 04:53 PM

Quote:

Originally Posted by epipko

I made a mistake pasting second line. It should read:

Code:

newest_dir=$(ssh root@rem_server ls -1rt /path/to/dir | tail -n 1)
scp -r root@rem_server:/path/to/$newest_dir /path/to/local_dir

You said that first line works for you, but it does not return anything back for me.

The first line is only a variable assignment. It's not supposed to produce any output. Have you tried to echo it? You'll want to develop some debugging skills for such problems. If something does not work, inspect the code and what it does step by step.
Issue the first command in a terminal, then check the content of variable $newest_dir with echo as suggested before.

If it is what you want it to be, move on to the second step. Does the scp command run if you hard-code the directory? And so on.
If you can't debug it yourself you'll need to give us some more info.

What output are you getting when running the commands? Any error messages? If so, post them please. If you do not want to post the actual directory names, please create dummy directories / files for testing, and post the real output of the commands you run.

This is to avoid that while anonymizing the commands and their output you introduce differences which are obscuring relevant information. (As just happened with the second command)

joe_2000 · 03-16-2015, 05:30 PM

Quote:

Originally Posted by unSpawn

http://mywiki.wooledge.org/ParsingLs ergo +1 for using "date --date='1 day ago'".

Some very good points made in that link. You'll definitely want to take a look! Thanks for sharing this unSpawn!

epipko · 03-17-2015, 12:37 AM

Thank you very much for your help and guidance. It works and I've learn something on the way. As far as unSpawn's link, I think we'll be ok using your solution as directory names are programmatically made, so we won't have weird cases described in the article. Thanks again.

ehdf · 03-17-2015, 06:29 AM

at first i was posting the same question into a new thread but thank God i got it resolved here.

joe_2000 · 03-17-2015, 04:12 PM

Quote:

Originally Posted by epipko

Thank you very much for your help and guidance. It works and I've learn something on the way. As far as unSpawn's link, I think we'll be ok using your solution as directory names are programmatically made, so we won't have weird cases described in the article. Thanks again.

Glad you made it work. Please mark the thread as solved if you consider your question as answered.