LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-24-2020, 07:13 AM   #1
orangepeel190
Member
 
Registered: Aug 2016
Posts: 64

Rep: Reputation: Disabled
wget/curl file that has been uploaded within the last 2-3 days?


Hi all,

I am seeking thought what line I could add to a download script. I am seeking to only download $file that was uploaded/modified within the last 2-3 days?

The reason is that every week, the same filename to $file so when my script seeks out this file, it has no idea if the file is the latest version (modified/uploaded within the last few days) or if it is the previous/old version of the file.

I am thinking of using something like:

find . -mtime -2 for two days prior

Then there is ls -ltr but I am struggling to work out what to grep/pipe as output to ensure that it only downloads the latest file (within the last 2-3 days_ when the script runs

the download command is similar to:

Code:
  echo "Getting file from Webserver"
  /usr/bin/curl -SkLo $file http://some.webite/here
I am having difficulties is working out what commands put in place to verify that the file that is about to be downloaded has been uploaded in the past 2-3 days. I am assuming this "magic line" would go after the curl line .....curl -SkLo $file http://some.webite/here | (HERE)

I am reading something about curl --time-cond or -z. Will that check the date details of the file on the server (to be potentially downloaded) or simply compare against an existing file?

Am I on the right path?
 
Old 07-24-2020, 07:37 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,444
Blog Entries: 3

Rep: Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711
You'd need to try tweaking the HTTP Request headers so that it sends an if-modified-since field along with the actual request. Both wget and curl can do that, but you'll have to dig in the manual pages. However, not all servers will respect that header.
 
Old 07-24-2020, 07:45 AM   #3
orangepeel190
Member
 
Registered: Aug 2016
Posts: 64

Original Poster
Rep: Reputation: Disabled
Hi Turbo. Thanks for your reply.

What about something similar to:

Code:
findme="`date --date="-3 days ago"`-restoffilenamehere"
file="alwaysthesamename"

curl -SkLoz $findme $file http://some.webite/here
Would that best the best way to get the file name (to compare to - 3 days earlier) into the download curl scripting or correct syntax? I am unable to get this to work....
 
Old 07-24-2020, 07:55 AM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,444
Blog Entries: 3

Rep: Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711Reputation: 2711
Untested:

Code:
curl --header "If-Modified-Since: $(date -d '-3 days' -u +'%a, %d %b %Y %T GMT')" \
        -SkLoz http://some.webite/here/$file
See "man curl" and mind the quotes and see the link in the previous post.
 
1 members found this post helpful.
Old 07-24-2020, 11:40 PM   #5
orangepeel190
Member
 
Registered: Aug 2016
Posts: 64

Original Poster
Rep: Reputation: Disabled
Thanks Turbo!!!

Ive implemented the correct syntax (thanks to your help) into my script and it adopted it to work as desired!

Appreciate your assistance!!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Display the list of items on webpage, that have not been updated in the last 3 days using msql sathishncd Programming 16 12-16-2017 09:04 AM
CentOS 6.7 has really OLD curl. Best way to update curl? sneakyimp Linux - Server 4 04-26-2016 04:06 PM
[SOLVED] Command to find all of the files which have been accessed within the last 30 days sunilup Linux - Newbie 1 05-16-2010 02:32 AM
rsync - sync files modified within the last 60 days boomklever Linux - Networking 3 06-03-2009 10:41 AM
cURL: Server has many IPs, how would I make a cURL script use those IPs to send data? guest Programming 0 04-11-2009 12:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:07 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration