LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   how to make wget user sytem time (https://www.linuxquestions.org/questions/linux-software-2/how-to-make-wget-user-sytem-time-494847/)

muxman 10-23-2006 09:10 AM

how to make wget user sytem time
 
How can I make wget use the current system time for the file date and time when it downloads a file? I don't want the date and time that the file has on it's server. I want the date and time I downloaded the file.

farslayer 10-23-2006 09:52 AM

you can touch the file after downloading it to change the time...

Quote:

itg-debian:/home/default# ls -al photoshare-4.2.0.zip
-rw-r--r-- 1 default default 321138 Jun 20 2005 photoshare-4.2.0.zip

itg-debian:/home/default# touch photoshare-4.2.0.zip

itg-debian:/home/default# ls -al photoshare-4.2.0.zip
-rw-r--r-- 1 default default 321138 Oct 23 10:50 photoshare-4.2.0.zip

timmeke 10-23-2006 09:55 AM

Why do you need those time stamps? Any particular reason?

I don't know if wget supports that (it would surprise me, since it doesn't seem logical to me).

Anyway, I'd go with farslayer's suggestion to use "touch".

b0uncer 10-23-2006 10:07 AM

Yup, it's an easy way. For an example:
Code:

wget http://someplace.com/somedir/filename_to_get && touch filename_to_get
Or, you could just create a very small scriptlet that takes arguments and passes them to wget, and after that touches the wget'ed file; make that script executable (named like wgett for example) and put it in /usr/bin or some place of your choice. A little more effort the first time, but makes things easier if you do a lot of this.

EDIT: wget seems to have somekind of "execute" option, which might enable you to use touch in conjunction with the wget command itself; with -N option to wget you can (from the man page) enable timestamping, though I'm not sure if this does what you want -- I didn't test it.

I might try wget -N first, then if it doesn't work, the -e (execute) switch and after that piping with touch.

muxman 10-23-2006 07:07 PM

Quote:

Originally Posted by timmeke
Why do you need those time stamps? Any particular reason?

I don't know if wget supports that (it would surprise me, since it doesn't seem logical to me).

Anyway, I'd go with farslayer's suggestion to use "touch".


It's so that when the files are listed by time, the files downloaded last have he newest date and time on them. For sorting files with similar names, like 11560049 and 11570123 and so on. A directory with 200 files like that can be hard to sort namewise, to the eye. But listed by date and time they are easier to see what data is coming in from the remote location and what files are the newest and most recent.

timmeke 10-24-2006 01:15 AM

To ease the sorting by name, you could use something like:
Code:

ls | sort -n
sort -n orders the entries by their numerical value (ascending), use -r if you want to reverse the order (ie descending).
Numerical sorting is safer than the alphabetical ordering done by "ls", in cases where the filenames consist only of numbers, like your examples.

If I understand you correctly, you want to be able to see which files were downloaded most recently, right? This would indeed mean that you can't just "touch" all your files each time wget finishes, because this would overwrite all last mod times ie making all files appear "equally recent".
Instead, you would only like to change the timestamps of the newly downloaded files.

For this, I propose the following solution:
1. Turn on wget's timestamping (-N) to make sure it downloads only new or updated files from the remote server.
2. Use wget's -o option to write the log to a file. You may also want to use -nv (non-verbose) to keep the amount of log information limited.
3. Use standard tools like grep, cut, awk, sed or simple scripting (ie Perl) to parse the log file.

I used a similar parsing method once, but don't have the script at hand here, unfortunately.
If I remember correctly though, you could simply "grep saved /your/log/file" to get a list of all saved (downloaded files). But this could be different now, since I was using a rather old version of wget at the time.

So, a good approach would be to make wget run with -N and -o (and possibly -nv), then taking a look at the log file created to see which lines you need and then trying to filter these lines out by using grep or awk. Finally, you may want to select the parts of those lines that represent the actual filenames, using tools like cut or sed.
Make sure to test this thoroughly.

Once you get a list of retrieved files from parsing the log, you can simply call "touch" on each of those files via:
Code:

for i in `cat filelist`; do
  touch ${i};
done;

or something similar.


All times are GMT -5. The time now is 11:53 PM.