LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-10-2017, 05:13 PM   #1
breadbin
Member
 
Registered: Sep 2002
Location: Dublin, Ireland
Distribution: Linux Mint 17
Posts: 129

Rep: Reputation: 25
command 'cp' acting weird - what's causing it?


I had a look at the cp man page and I still cannot figure it out. It probably is something stupid or noob that I haven't thought of but if I type the command

cp -R * source dest

how does it copy the files? As in how does it decide which order they are copied in.

I am trying to copy about 20 folders onto a usb stick. I can copy them no bother. I was using Nemo in Linux Mint 17 and it works perfectly for most situations. The problem occurs when I try and play the files on my car stereo. For some reason the car stereo orders the tracks as they are copied onto the stick which is weird. So when I play a folder it starts off with what seemed like some random file instead of track 1. And the rest of the tracks were all mixed up in order. Not very important but annoying!

I thought it was the tags until I ruled that out. Then I thought it was spaces in the filenames and I ruled that out too. I decided then to copy the files manually and individually onto the usb stick and they copied fine, played fine and in the right order. So when I made the folder myself with mkdir, copied them individually with cp it worked perfectly.

So then I thought it was Nemo, so I copied them with the CP command with the '*' wildcard thinking it would be hunky dory but with the -v parameter it shows the files being copied in what seems to be a random manner. The files are named with a zero padded number at the start to display them in the right order e.g.

01 - I Remember Now.mp3
02 - Anarchy-X.mp3
03 - Revolution Calling.mp3

etc.

So when I use the list command 'ls -l' it shows them in the right order. So when the car stereo seemed to be playing the files in a random order it was in fact playing them in the order that cp copied them. It started with 05, 02, 09, 10, 03 etc.

So my question is. Is there some file attribute that CP uses to pick the first file? Or is there a way to force it to copy alphabetically? It shouldn't make any difference for 99% of the time to be fair but it is super annoying.

I realise it's not really a noob quesiton but maybe someone can clear it up. I guess i can use the find command to exec the cp command as a quick workaround.

Oh and just to say the USB stick is formatted as FAT32.

Last edited by breadbin; 03-10-2017 at 05:15 PM.
 
Old 03-10-2017, 05:47 PM   #2
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,360

Rep: Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001
Quote:
Originally Posted by breadbin View Post
I had a look at the cp man page and I still cannot figure it out. It probably is something stupid or noob that I haven't thought of but if I type the command

cp -R * source dest

how does it copy the files? As in how does it decide which order they are copied in.
That command probably does not do what you intended. First, your shell will expand the "*" to a list of all the files and subdirectories in your current working directory. So, the cp command will first copy all of those, recursing into any subdirectories, and then start recursing into the "source" directory. All of that will end up in the "dest" directory.
Quote:
I am trying to copy about 20 folders onto a usb stick. I can copy them no bother. I was using Nemo in Linux Mint 17 and it works perfectly for most situations. The problem occurs when I try and play the files on my car stereo. For some reason the car stereo orders the tracks as they are copied onto the stick which is weird. So when I play a folder it starts off with what seemed like some random file instead of track 1. And the rest of the tracks were all mixed up in order. Not very important but annoying!
Most filesystems do not store directory entries in sorted order. For primitive directories like FAT, each new file is placed in the first empty slot, so reading the directory will return the names largely in the order they were placed there, though deleting a file and re-using that slot for another name can affect that. More advanced directories store the names in a database, and simply reading the names sequentially can show them in an unpredictable order, though at least one filesystem (and I forget which) does return the names in alphanumeric sequence.
Quote:
So then I thought it was Nemo, so I copied them with the CP command with the '*' wildcard thinking it would be hunky dory but with the -v parameter it shows the files being copied in what seems to be a random manner.

Oh and just to say the USB stick is formatted as FAT32.
That's because the "*" isn't being expanded where you think it is. Try
Code:
cp source/* dest
It's not going to descend into any subdirectories of "source", but it should copy the files in sequence (your shell did the sorting). If the FAT32 directory was previously empty, then that's the order in which they should appear.

I don't know of any way to ensure order when using "cp" recursively. There is no sorting available when it recurses into a directory.
 
Old 03-10-2017, 05:51 PM   #3
breadbin
Member
 
Registered: Sep 2002
Location: Dublin, Ireland
Distribution: Linux Mint 17
Posts: 129

Original Poster
Rep: Reputation: 25
well i don't know what happened but after googling 'cp alphabetical linux' etc. it seems this is a fairly common occurrence. I thought I had a good look for an answer but didn't. It's not a linux specific or 'cp' specific but just the way the car stereo reads the usb stick. i have a workaround by using 'rsync -av' but i would rather not copy everything via the command line.

anyone know is there a way to resort the files after i copy them with Nemo or can I change the behaviour of Nemo to copy them alphabetically?
 
Old 03-10-2017, 06:19 PM   #4
breadbin
Member
 
Registered: Sep 2002
Location: Dublin, Ireland
Distribution: Linux Mint 17
Posts: 129

Original Poster
Rep: Reputation: 25
thanks rknichols, that command worked great. It copied them over alphabetically. I still don't understand the reason behind it though. Is it to do with the time the original files were copied? Is there a way to edit the timestamp or is it the physical location of the file on the disc? So then if I move them to another folder alphabetically will they be sorted?
 
Old 03-10-2017, 06:30 PM   #5
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,360

Rep: Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001
Your shell is expanding the "source/*" to a list of everything in that directory, "source/01 - I Remember Now.mp3", "source/02 - Anarchy-X.mp3", "source/03 - Revolution Calling.mp3", ... . The shell always performs an alphabetical sort of the result from pathname expansion, so the "cp" command sees the names in alphabetical order.

Last edited by rknichols; 03-10-2017 at 06:32 PM.
 
Old 03-11-2017, 08:06 AM   #6
breadbin
Member
 
Registered: Sep 2002
Location: Dublin, Ireland
Distribution: Linux Mint 17
Posts: 129

Original Poster
Rep: Reputation: 25
I tried the cp command for the whole music folder but like you mentioned it sorts the folders alphabetically but not the files inside.

cp -v -R /source/* /dest/

i will probably have to learn how to write a script to copy the whole lot or copy the folders one by one.

On a different note I did use rsync -var to copy them over onto the usb stick but unfortunately I will have to copy them all over in one go because adding folders and files later will mean no more alphabetically sorting. I presume this goes for the cp script if I ever get round to making it. Might be easier to get a new stereo!
 
Old 03-11-2017, 10:19 AM   #7
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,360

Rep: Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001
As I said, I don't know of any way to ensure order when using "cp" recursively. Re-sorting the destination directory once the files are already there is going to be a mess unless there is some tool to do that specifically, and I don't know of one.

If the files are in various subdirectories, I believe the simplest way to handle it would be to create a temporary directory with symlinks to all the files, use "cp -L tmpdir/* /dest" to copy them, and then delete the temporary directory. Something like this should work:
Code:
#!/bin/sh
mkdir -p /tmp/musictemp
find /source -type f -exec ln -s -t /tmp/musictemp {} +
cp -L /tmp/musictemp/* /dest
rm -r /tmp/musictemp
Note that "/source" must be an absolute path (begins with "/") or the symlinks will not be valid. The "-L" option on the cp command will cause it to follow the symlinks and copy the actual files. Note that the ln command will complain about any duplicate names.
 
Old 03-11-2017, 02:52 PM   #8
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 13,390
Blog Entries: 9

Rep: Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654Reputation: 3654
Quote:
Originally Posted by breadbin View Post
cp -v -R /source/* /dest/
NO!
if you use the '-R' option, you don't need the wildcard '*'!!!
that is what wreaks havoc with your stuff, because the shell tries to expand it BEFORE the cp command reads it!
 
Old 03-11-2017, 05:50 PM   #9
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,360

Rep: Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001
Quote:
Originally Posted by ondoho View Post
NO!
if you use the '-R' option, you don't need the wildcard '*'!!!
The source directory contains subdirectories, so recursion is required. The problem is that alphabetical order is not preserved during recursion, and that's going to be true in any filesystem if there are multiple subdirectories since each subdirectory is going to be processed completely before moving on to the next. You need to get all the filenames in one place so that they can be sorted.
 
Old 03-11-2017, 11:19 PM   #10
descendant_command
Senior Member
 
Registered: Mar 2012
Posts: 1,716

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
cp can't remember how it started.
It just remembers doing what you told it ... told it ... told it ...

Last edited by descendant_command; 03-11-2017 at 11:20 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
OS acting weird MTK358 Linux - Newbie 5 01-07-2010 06:08 PM
[SOLVED] computer acting weird baronobeefdip Linux - Desktop 13 10-24-2009 09:29 PM
cdrom acting weird Vindane Slackware 5 05-30-2004 01:53 AM
Ping command is acting weird xailer Linux - Newbie 2 11-17-2003 10:37 AM
IPCHAINS is acting weird Snabber Linux - Networking 11 08-08-2003 11:07 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:57 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration