Copy a directory into another directory while ignoring specific directories or files

wh33t · 05-14-2012, 03:53 PM

Hey LQ,

I've been racking my brains for a few hours on this one and I'm pretty fed up. I turn to the pro's now.

This is for my clients webserver. I want to be able to mirror their entire live public_html/ directory into a different directory that is also located inside the public_html/ directory.

So something like:

Code:

/public_html/ to /public_html/destination_directory/

So the problem is that I want to ignore /public_html/destination_directory/ because I don't want to copy itself into itself if that makes sense.

I've been reading up on the "cp" command but it doesn't really seem like it can do this. I've tried doing things like

Code:

cp !(/public_html/destination_directory) /public_html/ /public_html/destination_directory

But that doesn't work either. Of the few examples I've looked at I can't understand them to make them work for my specific example. So I need some help.

I want to specifically ignore multiple directories and files and I will be running this command through the PHP function system(), so I'm not sure what terminal equivalent that would be. The VPS Webhost is CentOS if that helps at all.

Thanks so much in advance!

Tinkster · 05-14-2012, 04:17 PM

You're almost there ...
first, make sure extglob is on for this activity:
shopt -s extglob
Then:
cd public_html
cp !(destination_directory) destination_directory

wh33t · 05-14-2012, 04:55 PM

Quote:

Originally Posted by Tinkster

You're almost there ...
first, make sure extglob is on for this activity:
shopt -s extglob
Then:
cd public_html
cp !(destination_directory) destination_directory

Remember, I'm running this from system() inside of a PHP script. I'm not sure what "shopt -s extglob" does exactly but does it permanently alter the way the server runs or is it just for the terminal session or script duration? Do I run it in the same system() exec or do I run a system('shopt -s extglob') first?

Btw, thank you for chiming in.

Tinkster · 05-14-2012, 06:41 PM

Quote:

Originally Posted by wh33t

Remember, I'm running this from system() inside of a PHP script. I'm not sure what "shopt -s extglob" does exactly but does it permanently alter the way the server runs or is it just for the terminal session or script duration? Do I run it in the same system() exec or do I run a system('shopt -s extglob') first?

Btw, thank you for chiming in.

That should be just for the current shell you're invoking; it's not a "system setting".

You'd have to find a way of doing this from the same session, so maybe slap
shop -s extglob and your cp into a script, and run THAT from your system() call.

Cheers,
Tink

wh33t · 05-14-2012, 06:47 PM

Quote:

Originally Posted by Tinkster

That should be just for the current shell you're invoking; it's not a "system setting".

You'd have to find a way of doing this from the same session, so maybe slap
shop -s extglob and your cp into a script, and run THAT from your system() call.

Cheers,
Tink

Is there a way to run both in the same system() call?

Something like:

Code:

system('shop -s extglob; cp !ignoresomestuff source_dir destination_dir';)

Tinkster · 05-14-2012, 07:01 PM

Quote:

Originally Posted by wh33t

Is there a way to run both in the same system() call?

Something like:

Code:

system('shop -s extglob; cp !ignoresomestuff source_dir destination_dir';)

Yah.

Code:

$ cat test.php
<?php
  $out=system('ls -1 *pl;echo "It is working!" ');
  print $out;
?>

Code:

php -f test.php 
iotest.pl
test.pl
It is working!
It is working!

Not that I understand why I get the echo's output twice, but I'm not a PHP guy ;}

Cheers,
Tink

wh33t · 05-14-2012, 07:16 PM

Quote:

Originally Posted by Tinkster

Yah.

Code:

$ cat test.php
<?php
  $out=system('ls -1 *pl;echo "It is working!" ');
  print $out;
?>

Code:

php -f test.php 
iotest.pl
test.pl
It is working!
It is working!

Not that I understand why I get the echo's output twice, but I'm not a PHP guy ;}

Cheers,
Tink

You're awesome man.

Now, one last question. If I want to ignore multiple files do I use commas?

Example:

Code:

cp !(file1,file2,directory1,directory2) source destination

You get the output twice because system() by design echo's the output. If you just wanted only the output from the commands then you would as you have done by called it by exec() or passthru() which do not output anything.

Edit:

I have just tried running this:

Code:

shopt -s extglob; cp !test_file.txt /home/admin/public_html/dev/test/source_dir/* /home/admin/public_html/dev/test/destination_dir

And it still copies over !test_file.txt. I've tried with brackets around test_file.txt and it didn't copy anything at all. Yet doesn't seem to error on me either.

Tinkster · 05-14-2012, 08:49 PM

I think I know what you're doing wrong ;}

for the glob to work you need to be in the source dir.
shopt -s extglob; cd /home/admin/public_html/dev/test/source_dir/; cp !(test_file.txt) /home/admin/public_html/dev/test/destination_dir

wh33t · 05-14-2012, 09:01 PM

So... if you do "cp" from inside of a dir with out specifying (*) it just assumes every file and directory in the cwd?

Tinkster · 05-14-2012, 09:08 PM

Everything that doesn't match what you have in !() ... that's the whole idea of that glob.

wh33t · 05-14-2012, 09:19 PM

Code:

shopt -s extglob; cd /home/admin/public_html/dev/test/source_dir/; cp !(test_file.txt) /home/admin/public_html/dev/test/destination_dir

Unfortunately doesn't work. I think it's because the terminal type must be different when running it from PHP. I'm having some luck with another solution I've googled like this:

Code:

find /home/admin/public_html/dev/test/source_dir/  ! -name "test_file.txt" | cpio -admp home/admin/public_html/dev/test/destination_dir

Which does work however in the $destination_dir it creates the total directory path from the source. So in the destination directory I get the directories made like:

Code:

/home/admin/public_html/test/source_dir/test_file_2.txt

Which is nearly what I want. It's ignoring the file like it should but now it's making directories.

chrism01 · 05-15-2012, 12:27 AM

How about a local rsync using exclude option http://www.thegeekstuff.com/2011/01/...s-and-folders/

wh33t · 05-15-2012, 12:19 PM

Quote:

Originally Posted by chrism01

How about a local rsync using exclude option http://www.thegeekstuff.com/2011/01/...s-and-folders/

Holy @#$%, that was very easy to do. Thank you very much.

I only have one issue... Every time I run my back up command it changes the file permission of my destination directory to 750 when it's supposed to be set at 755... Any idea why that might be?

chrism01 · 05-16-2012, 07:44 PM

Is that using rsync? If so, checkout the various flags available.
I usually go with -a

Quote:

-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)

http://linux.die.net/man/1/rsync
Basically, those implied flags preserve just about everything ...

wh33t · 05-16-2012, 08:13 PM

Rsync is running from PHP on my VPS. I do a manual CHMOD from PHP to get around the issue and it works great. Thanks for your help!