LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-14-2015, 10:52 AM   #1
atjurhs
Member
 
Registered: Aug 2012
Posts: 190

Rep: Reputation: Disabled
finding in wrong order so cat is messed up


hi guys,

thanks in adv for looking through the long winded post...

i'm trying to cat a bunch of text files together into one, but i'm running into problems and i think it's because of how 'find' is working, but i'm not really sure. here's the command i'm using


Code:
find . -name '*.dat' -exec cat {} \; > out.txt
my data structure looks like

dir1
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir2
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir3
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir4
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir5
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir6
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir7
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir8
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir9
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat
dir10
f1.dat
f2.dat
f3.dat
f4.dat
f5.dat
f6.dat
f7.dat
f8.dat
f9.dat
f10.dat

so I "think" in my cat'd file I'm getting the order like this,

dir1f1
dir1f10
dir1f2
dir1f3
dir1f4
dir1f5
dir1f6
dir1f7
dir1f8
dir1f9
dir2f1
dir2f10
dir2f2
dir2f3
dir2f4
dir2f5
dir2f6
dir2f7
dir2f8
dir2f9
etc....

but maybe it's even worse than this, idk

Last edited by atjurhs; 01-16-2015 at 01:25 PM. Reason: clarified some goof ups in how the output is
 
Old 01-14-2015, 11:12 AM   #2
veerain
Senior Member
 
Registered: Mar 2005
Location: Earth bound to Helios
Distribution: Custom
Posts: 2,524

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
Code:
for d in `find . -maxdepth 1 -mindepth 1 -type d | sort -V`; do
 for f in `find $d -maxdepth 1 -mindepth 1 -type f -name "*.dat" | sort -V`; do
  cat $f >> out_file
 done
done
This may help.
 
Old 01-14-2015, 08:23 PM   #3
Fred Caro
Member
 
Registered: May 2007
Posts: 999

Rep: Reputation: 166Reputation: 166
What happens if you put the curly brackets in single quotes?


Fred.
 
Old 01-14-2015, 10:55 PM   #4
veerain
Senior Member
 
Registered: Mar 2005
Location: Earth bound to Helios
Distribution: Custom
Posts: 2,524

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
In the find command you should put curly bracket in either single or double quotes. Otherwise it would be interpreted by the shell and not find command.
 
Old 01-14-2015, 11:23 PM   #5
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,401

Rep: Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489
Quote:
Originally Posted by veerain View Post
In the find command you should put curly bracket in either single or double quotes. Otherwise it would be interpreted by the shell and not find command.
Makes no difference in this case. The shell is smarter than that.

The output from find is unsorted, just the order in which it encounters the entries in each directory, similar to what you get if you run "ls -U" to get unsorted output. Some filesystems do store directories in a manner that keeps the names sorted. Most do not. If you want sorted output, pipe the result through sort.
 
Old 01-15-2015, 02:39 AM   #6
coltree
Member
 
Registered: Nov 2003
Location: Jacobs Well, Queensland AU
Distribution: Slackware
Posts: 68
Blog Entries: 1

Rep: Reputation: 9
Your naming convention is wrong.
f01.dat, etc
dir01, etc
if the system will grow to hundreds of directories or dat files
f001.dat, etc
dir001, etc
same idea if you want to sort by date-time
20140115-1835

a good rule of programming - the earlier you fix a problem the easier it is to fix
corollary - the more times you try and fix it the worse it gets
 
Old 01-15-2015, 03:30 AM   #7
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,220

Rep: Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697Reputation: 2697
you did not specify the order you want. I think find will not sort/reorder the files it found, that is the order of appearance. If you need something else you need to implement that.
 
Old 01-15-2015, 08:40 PM   #8
cfajohnson
LQ Newbie
 
Registered: Aug 2012
Distribution: Linux Mint 17
Posts: 22

Rep: Reputation: Disabled
Quote:
Originally Posted by veerain View Post
Code:
for d in `find . -maxdepth 1 -mindepth 1 -type d | sort -V`; do
 for f in `find $d -maxdepth 1 -mindepth 1 -type f -name "*.dat" | sort -V`; do
  cat $f >> out_file
 done
done
This may help.
An execellent example of how not to use find (or almost any other command).
 
Old 01-15-2015, 08:44 PM   #9
cfajohnson
LQ Newbie
 
Registered: Aug 2012
Distribution: Linux Mint 17
Posts: 22

Rep: Reputation: Disabled
Quote:
Originally Posted by veerain View Post
In the find command you should put curly bracket in either single or double quotes. Otherwise it would be interpreted by the shell and not find command.
Nonsense. The quotes make no difference.
 
Old 01-15-2015, 08:49 PM   #10
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

from your post it's not clear what you actually want or what the directory structure is, but if I'm guessing correctly, it could be as simple as:
Code:
cat */*.dat > out.txt
You can confirm the order the files will be cat'd using echo. Eg
Code:
echo */*.dat
Or perhaps for enhanced readability:
Code:
echo */*.dat | tr ' ' '\n'
If this does not do what you want please clearly state the directory structure is and what you want to achieve.

Evo2.
 
Old 01-15-2015, 08:55 PM   #11
cfajohnson
LQ Newbie
 
Registered: Aug 2012
Distribution: Linux Mint 17
Posts: 22

Rep: Reputation: Disabled
If you want f10 to sort after f9, then you have to pad 9 to 09.

However, find does not sort files alphabetically; it prints them in the order they are listed in the directory. Shell expansion does sort files alphabetically.

Code:
cat dir*/*.dat > "$file"
If that gives an "Argument list too long" error, proceed directory by directory:

Code:
for dir in dir*
do
  cat "$dir"/*.dat
done > "$file"

Last edited by cfajohnson; 01-15-2015 at 08:56 PM.
 
Old 01-15-2015, 08:59 PM   #12
cfajohnson
LQ Newbie
 
Registered: Aug 2012
Distribution: Linux Mint 17
Posts: 22

Rep: Reputation: Disabled
Quote:
Originally Posted by evo2 View Post
Hi,
Code:
echo */*.dat | tr ' ' '\n'
There's no need for tr:
Code:
printf "%s\n" dir*/*.dat
 
Old 01-15-2015, 09:14 PM   #13
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Quote:
Originally Posted by cfajohnson View Post
There's no need for tr:
There's no *need* for printf either. Assuming the OP is using bash, printf is not a shell builtin, a new process is spawned either way. Indeed there are many way to get that output, feel free to post as many of them as you like.
 
Old 01-15-2015, 09:35 PM   #14
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,688

Rep: Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259
In some cases, you will find it better to use a file than trying to get all the find loops, and sort loops in the proper order.

Find the files and put them in a file... Now you can try various sorts to try and get them in the order you want. When the file is the way you want (or you can just edit one that is close and force it , then use the file as input (for i in $(cat file); do..., or better, while read f; do... done<file)

For future reference, it is easy to sort file names in alphabetical order. That is why the suggestions to use the form "name.nnn" (or similar) where the nnn is the number of the file with leading zeros. This allows a simple sort to work easily.
 
Old 01-15-2015, 09:39 PM   #15
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,
Quote:
Originally Posted by jpollard View Post
For future reference, it is easy to sort file names in alphabetical order. That is why the suggestions to use the form "name.nnn" (or similar) where the nnn is the number of the file with leading zeros. This allows a simple sort to work easily.
If using the sort command from coreutils you can use the -V option to get the desired results without leading zeros.

Evo2.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Oops, cat iso file to wrong device drivekiller361 Linux - General 21 01-02-2015 02:43 PM
[SOLVED] MBR, GRUB and Partitions messed up : Partition table entries are not in disk order fabien Linux - Laptop and Netbook 6 04-12-2012 01:37 PM
[SOLVED] concat files in numerical order using cat and sort DEF. Programming 16 01-27-2010 02:53 PM
cat safely without messed up terminal curos Slackware 6 08-20-2006 01:54 PM
logrotate log files - keeping in order when using grep or cat and bash dmellem Linux - Software 6 02-11-2005 01:05 PM


All times are GMT -5. The time now is 02:27 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration