LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 12-08-2017, 05:44 AM   #1
atux_null
Member
 
Registered: Mar 2015
Distribution: debian 12, ubuntu 22.04
Posts: 65

Rep: Reputation: Disabled
sync remote to local machine


hi. there a server on the net that has an IP 3.3.3.3 and i have only https access with username and password. The site simply has a directory listing and it has a few folders(A,B,...,Z). I do need only 2 of them (A, D) to get them in a local machine with IP 192.168.1.5 and serve them through apache. The server gets updated at random times >4 times a day and the files are >1GB. The servers deletes the old files and keeps only the latest ones.
now we get the folders and their files manually using wget. i would like to automate it and have a script running every 30 minutes and check for differences. If it finds a newer file in folder A, delete all the local contents and download the newer. Repeat the same process for folder D.
at the moment i have
Code:
#!/bin/bash
wget -N --no-check-certificate --user atux password null -r -np https://3.3.3.3/D -P /var/www/html
wget -N --no-check-certificate --user atux password null -r -np https://3.3.3.3/A -P /var/www/html
there are two problems:
-it downloads in the local server, everything under /var/www/html/3.3.3.3/A and D respectively. i would like to have it under /var/www/html//A and D respectively, without the remote IP 3.3.3.3
-it keeps downloading the newer files, without deleting the older ones.

could someone help me over with the script please?
 
Old 12-08-2017, 02:01 PM   #2
HappyTux
Senior Member
 
Registered: Mar 2003
Location: Nova Scotia, Canada
Distribution: Debian AMD64
Posts: 4,170

Rep: Reputation: 244Reputation: 244Reputation: 244
Give rsync a go a basic command like rsync -avP -delete atux@3.3.3.3:/var/www/html/D /var/www/html/ run from the command line should sync the local D directory to the exact contents of the remote. It only copies the changes to the remote files and deletes the files on the local copy if they are not on the remote copy anymore. You would need to adapt the line to put it in script removing the vP from -avP as verbose output and resume where left off in interrupted transfer are not needed in one. Most likely will want paswordless sshd login on remote machine setup as well to allow the command to execute without the need to enter password.
 
Old 12-08-2017, 02:46 PM   #3
atux_null
Member
 
Registered: Mar 2015
Distribution: debian 12, ubuntu 22.04
Posts: 65

Original Poster
Rep: Reputation: Disabled
i am afraid i do not have ssh access. Only https access with password.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] SSH remote command: Pipe remote output to local machine? kenneho Linux - Server 6 12-06-2012 01:37 AM
mounting samba filesystem on remote machine as /home on local machine shishirkotkar Linux - Software 1 04-28-2008 05:05 AM
scp: copy a file from local machine to remote machine seran Linux - Newbie 8 10-30-2007 12:23 PM
sync local with remote binary_dreamer Linux - Software 3 05-04-2006 02:57 AM
Complicated one: MySQL, rsync, remote machine, local machine daiver Linux - General 2 02-27-2006 12:14 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:59 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration