LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-22-2017, 01:46 PM   #1
qkzoo
LQ Newbie
 
Registered: Jan 2012
Distribution: Lubuntu & Debian Squeeze
Posts: 12

Rep: Reputation: Disabled
SCP Playlist Script to local machine.


My goal is to have a script that reads a playlist file (.m3u), reads each line, then downloads that file to a folder on my local machine over SSH.

I found a script online that does this using cp instead of SCP, so I adapted it a little, but it's not working, and I'm not script savvy.

Code:
#!/bin/bash
grep -v '#' 5_Star.m3u | tr -d '\r' |while read line ; do scp pi@192.168.1.2:"/media/mypassport$line" "/home/andrew/temp/" ; done
It appears to me that the spaces in the filenames are breaking the script, as I see a lot of this:

Code:
scp: /media/mypassport/music/Collective: No such file or directory
scp: Soul: No such file or directory
...the folder name is "Collective Soul...", so it looks like it's splitting after each space?

Can anybody offer some additions to make this script work properly?
 
Old 10-22-2017, 02:11 PM   #2
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,235
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
test the script with

- adding 'set -x' just under #!/bin/bash

and

- prepending 'echo' before scp

show us.
 
Old 10-22-2017, 08:17 PM   #3
frankbell
LQ Guru
 
Registered: Jan 2006
Location: Virginia, USA
Distribution: Slackware, Ubuntu MATE, Mageia, and whatever VMs I happen to be playing with
Posts: 17,386
Blog Entries: 27

Rep: Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405Reputation: 5405
This link discusses ways of replacing the spaces in the file names with underscores using scripting: https://stackoverflow.com/questions/...-a-bash-script
 
Old 10-23-2017, 10:25 AM   #4
michaelk
Moderator
 
Registered: Aug 2002
Posts: 21,482

Rep: Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091
Although the hint about using set -x should provide some help on what might be happening it may not be obvious.

bash will strip the quotes before the scp command runs so spaces are not escaped on the server. Basically you need to double quote the spaces.

Without seeing the exact playlist try
Code:
scp pi@192.168.1.2:/media/mypassport/""$line"" /home/andrew/temp/
 
Old 10-23-2017, 08:17 PM   #5
qkzoo
LQ Newbie
 
Registered: Jan 2012
Distribution: Lubuntu & Debian Squeeze
Posts: 12

Original Poster
Rep: Reputation: Disabled
After running -set x and adding the echo, plus the quotes, here is what I get:

Quote:
+ read line
+ echo scp pi@192.168.1.2:/media/mypassport/music/The Pretty Reckless - Going To Hell/07 - The Pretty Reckless - Absolution.mp3 /home/andrew/temp/
scp pi@192.168.1.2:/media/mypassport/music/The Pretty Reckless - Going To Hell/07 - The Pretty Reckless - Absolution.mp3 /home/andrew/temp/
Looks like the spaces aren't escaped?
 
Old 10-23-2017, 08:39 PM   #6
michaelk
Moderator
 
Registered: Aug 2002
Posts: 21,482

Rep: Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091
Your correct I messed up my quotes...
Code:
scp pi@192.168.1.2:/media/mypassport/"'$line'" /home/andrew/temp/
 
Old 10-23-2017, 09:43 PM   #7
qkzoo
LQ Newbie
 
Registered: Jan 2012
Distribution: Lubuntu & Debian Squeeze
Posts: 12

Original Poster
Rep: Reputation: Disabled
That worked with the spaces. Some characters are still breaking, i.e., the single quote ' and the ampersand & broke quite a few track titles. Any suggestions on those?
 
Old 10-23-2017, 10:15 PM   #8
michaelk
Moderator
 
Registered: Aug 2002
Posts: 21,482

Rep: Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091
Post the exact path/title.

Sometimes as frankbell suggested it is easier to change the name...
 
Old 10-23-2017, 11:44 PM   #9
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,860

Rep: Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596
I always change the names of files like that, even if it's only temp for processing in a *nix system.
Otherwise you're always having to fool around with every cmd... just too hard.
I use underscores instead of spaces, remove quotes. Ampersands I think are safe if surrounded by other (non-whitespace) chars.
 
Old 10-24-2017, 01:32 AM   #10
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,235
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
is that a quirk of scp?
i really can't see anything wrong with the syntax used in post #1.
i'm sure it would have worked with a normal cp command?
 
Old 10-24-2017, 06:24 AM   #11
michaelk
Moderator
 
Registered: Aug 2002
Posts: 21,482

Rep: Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091Reputation: 4091
Does seem to be a "quirk" of ssh/scp and your correct it should of worked if just using a normal cp command.

Another option which might work better is to create a sftp batch file. While more complicated you should not have to deal with all the exceptions.
 
1 members found this post helpful.
Old 10-24-2017, 10:19 PM   #12
qkzoo
LQ Newbie
 
Registered: Jan 2012
Distribution: Lubuntu & Debian Squeeze
Posts: 12

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by chrism01 View Post
I always change the names of files like that, even if it's only temp for processing in a *nix system.
Otherwise you're always having to fool around with every cmd... just too hard.
I use underscores instead of spaces, remove quotes. Ampersands I think are safe if surrounded by other (non-whitespace) chars.
If I had started my library like that it would be an option, but I have over 400 albums I've scanned into my library and really don't want to have to go back and fix everything. I did find a solution here, check this out:

Code:
#!/bin/bash
# set -x
#grep -v '#' 5_Star.m3u | tr -d '\r' |while read line ; do scp pi@192.168.1.2:/media/mypassport"'$line'" /home/andrew/temp/ ; done
grep -v '#' 5_Star.m3u | tr -d '\r' |while read line ; do scp pi@192.168.1.2:/media/mypassport"$(printf '%q' "$line")" /home/andrew/temp/music/ ; done
Not exactly sure why it works, but it did. I had only 1 album that didn't go through, and still not sure why, as there were no funky characters in the filename, but since it was only 1, I was able to copy that over manually.

Would there be a way to tell this script to skip a file if it already exists on the local computer, just to jazz it up and make this script so I could use it to update my local files?
 
Old 10-24-2017, 11:11 PM   #13
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,860

Rep: Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596
It is fairly easy to script the name changes, and you only need to write that program once.
You can re-use it if/when you need it again.
(eg any time you bring in files from MSWin if you write it generically)
After running it, you'll never have to worry again for those (renamed) files.

Re your other qn, see rsync eg https://www.tecmint.com/rsync-local-...tion-commands/

Last edited by chrism01; 10-24-2017 at 11:12 PM.
 
Old 10-25-2017, 08:41 PM   #14
qkzoo
LQ Newbie
 
Registered: Jan 2012
Distribution: Lubuntu & Debian Squeeze
Posts: 12

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by chrism01 View Post
It is fairly easy to script the name changes, and you only need to write that program once.
You can re-use it if/when you need it again.
(eg any time you bring in files from MSWin if you write it generically)
After running it, you'll never have to worry again for those (renamed) files.

Re your other qn, see rsync eg https://www.tecmint.com/rsync-local-...tion-commands/
I might check out a solution for renaming. As for the rsync solution, is there a maximum for the number of includes you can use with it? My one playlist has 455 songs in it, so if I was piping each filename to a single rsync command it would be a very large command! My music library is not everything just all jumbled in a single folder, it's all organised by artist/album title and then every song for that album is in that folder, even ones I don't really listen to. I wouldn't want to rsync those to my local pc.
 
Old 10-26-2017, 01:03 AM   #15
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,235
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
personally, i can see the need for what qkzoo has scripted, and i would be against touching these filenames.
because new ones are added all the time (at least that's how it is with me), media servers rely on the names (and think there's new files if you rename them), the order of files (i.e. songs) is sometimes defined by those names etc...
since a usable workaround exists, i wouldn't touch the filenames.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Pipe output of local script, ran on remote machine, to local python script trendal Programming 17 07-04-2016 01:56 AM
[SOLVED] Trouble SCP'ing a file from remote server to local machine, running out of ideas! bowserface Linux - Newbie 17 02-19-2014 08:06 AM
[SOLVED] How can I ssh/scp into a remote's machine local machine? venom4u31 Linux - Newbie 6 11-05-2011 01:36 PM
scp: copy a file from local machine to remote machine seran Linux - Newbie 8 10-30-2007 12:23 PM
Build a local playlist to play remote audio files (through HTTP) BillyGalbreath Linux - Software 1 11-01-2006 11:25 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:58 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration