LinuxQuestions.org - sync remote to local machine

- Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)

- - sync remote to local machine (https://www.linuxquestions.org/questions/linux-general-1/sync-remote-to-local-machine-4175619186/)

sync remote to local machine

hi. there a server on the net that has an IP 3.3.3.3 and i have only https access with username and password. The site simply has a directory listing and it has a few folders(A,B,...,Z). I do need only 2 of them (A, D) to get them in a local machine with IP 192.168.1.5 and serve them through apache. The server gets updated at random times >4 times a day and the files are >1GB. The servers deletes the old files and keeps only the latest ones.
now we get the folders and their files manually using wget. i would like to automate it and have a script running every 30 minutes and check for differences. If it finds a newer file in folder A, delete all the local contents and download the newer. Repeat the same process for folder D.
at the moment i have

Code:

#!/bin/bash

wget -N --no-check-certificate --user atux password null -r -np https://3.3.3.3/D -P /var/www/html

wget -N --no-check-certificate --user atux password null -r -np https://3.3.3.3/A -P /var/www/html

there are two problems:
-it downloads in the local server, everything under /var/www/html/3.3.3.3/A and D respectively. i would like to have it under /var/www/html//A and D respectively, without the remote IP 3.3.3.3
-it keeps downloading the newer files, without deleting the older ones.

could someone help me over with the script please?

Give rsync a go a basic command like rsync -avP -delete atux@3.3.3.3:/var/www/html/D /var/www/html/ run from the command line should sync the local D directory to the exact contents of the remote. It only copies the changes to the remote files and deletes the files on the local copy if they are not on the remote copy anymore. You would need to adapt the line to put it in script removing the vP from -avP as verbose output and resume where left off in interrupted transfer are not needed in one. Most likely will want paswordless sshd login on remote machine setup as well to allow the command to execute without the need to enter password.

i am afraid i do not have ssh access. Only https access with password.