LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 01-22-2011, 04:08 PM   #1
spoovy
Member
 
Registered: Feb 2010
Location: London, UK
Distribution: Scientific, Ubuntu, Fedora
Posts: 373

Rep: Reputation: 33
How to list contents of all files recursively


What I would like to do is to print the contents of all text files in a particular directory, recursively. Problem being that there are directories and possibly binaries scattered around in the filesystem as well.

Trying cat * works as long as there are no directories in there, but when there are it gives an error instead and prints nothing.

I'm sure it's easy using file -f or something but I can't figure it!

Any ideas?

Thanks in advance

spoov
 
Old 01-22-2011, 04:32 PM   #2
AlucardZero
Senior Member
 
Registered: May 2006
Location: USA
Distribution: Debian
Posts: 4,824

Rep: Reputation: 615Reputation: 615Reputation: 615Reputation: 615Reputation: 615Reputation: 615
Code:
find /dir -type f -exec cat '{}' +;
?
 
1 members found this post helpful.
Old 01-23-2011, 10:26 AM   #3
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
catting some non-text files is likely to do strange things to the terminal; it would be prudent to run the file command on each file found and only cat the file if it is a text type. This could most easily be done by writing a script and calling that from find (beware that AlucardZero's +; has been changed to \; so the script is called with a single file name):
Code:
find /dir -type f -exec /path_to_your_script '{}' \;
The script could be something like this (not tested):
Code:
#!/bin/bash

case "$(file "$1" | sed 's/.*: //' ) in
    *'ASCII C'*text* \
        | *'HTML document text' \
        | *'ISO-8859'*text* \
        | *'M3U playlist text' \
        | *'UTF-8 Unicode'*text* \
        | *'Windows Registry text'* \
        | 'XML document text' \
        | *'a /bin/bash script text executable' \
        | *'exported SGML document text' \
            )
                cat "$1"
                ;;
    * )
        echo "Skipped (not a recognised text type): $1"
esac
It could be made more robust and could loop over multiple file names given as arguments but is probably "good enough" as long as you don't have a lot of files.

Last edited by catkin; 01-23-2011 at 10:27 AM. Reason: spurious text excised
 
1 members found this post helpful.
Old 01-23-2011, 02:30 PM   #4
spoovy
Member
 
Registered: Feb 2010
Location: London, UK
Distribution: Scientific, Ubuntu, Fedora
Posts: 373

Original Poster
Rep: Reputation: 33
Quote:
Originally Posted by AlucardZero View Post
Code:
find /dir -type f -exec cat '{}' +;
?

Excellent! Thanks v much, that worked for my purpose.

@Catkin - you're right about cat-ing non-text files possibly doing strange things, it buggered up my console on one occasion, causing the font to change to some non-ASCII gobbldigook.
Thanks very much for your additional info, i'll try that out.
 
Old 01-23-2011, 02:43 PM   #5
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
Quote:
Originally Posted by spoovy View Post
@Catkin - you're right about cat-ing non-text files possibly doing strange things, it buggered up my console on one occasion, causing the font to change to some non-ASCII gobbldigook.
In that case you can type reset, even if you cannot read what you type, press enter and the terminal restores its normal status.
 
Old 01-23-2011, 03:10 PM   #6
spoovy
Member
 
Registered: Feb 2010
Location: London, UK
Distribution: Scientific, Ubuntu, Fedora
Posts: 373

Original Poster
Rep: Reputation: 33
Catkin - What do you mean by "(beware that AlucardZero's +; has been changed to \; so the script is called with a single file name):"?

I don't know what the + character does in the first place tbh!
 
Old 01-23-2011, 03:27 PM   #7
AlucardZero
Senior Member
 
Registered: May 2006
Location: USA
Distribution: Debian
Posts: 4,824

Rep: Reputation: 615Reputation: 615Reputation: 615Reputation: 615Reputation: 615Reputation: 615
man find -
Code:
      -exec command ;
              Execute command; true if 0 status is returned.  All following arguments to find are
              taken to be arguments to the command until an argument consisting of ‘;’ is encoun‐
              tered.  [..cut..]

       -exec command {} +
              This  variant of the -exec action runs the specified command on the selected files,
              but the command line is built by appending each selected file name at the end;  the
              total  number  of  invocations  of the command will be much less than the number of
              matched files. [..etc..]
It's the difference between
Code:
cat a
cat b
cat c
and
Code:
cat a b c
 
Old 01-24-2011, 05:58 AM   #8
Manjunath1847
Member
 
Registered: Mar 2007
Posts: 44

Rep: Reputation: 16
cat `find . -name "*" | xargs file | grep ASCII | cut -d ":" -f1`

Quote:
Originally Posted by spoovy View Post
What I would like to do is to print the contents of all text files in a particular directory, recursively. Problem being that there are directories and possibly binaries scattered around in the filesystem as well.

Trying cat * works as long as there are no directories in there, but when there are it gives an error instead and prints nothing.

I'm sure it's easy using file -f or something but I can't figure it!

Any ideas?

Thanks in advance

spoov
 
Old 01-24-2011, 11:19 AM   #9
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Quote:
Originally Posted by Manjunath1847 View Post
cat `find . -name "*" | xargs file | grep ASCII | cut -d ":" -f1`
A good start but a few issues with that.

In find . -name "*", -name "*" gives the default behaviour so could as well be find . And spoovy has said only ordinary files are relevant -- not directories or any of the special file types -- so we can cut down on the files passed down the pipeline (some of which, in perverse cases, could cause unwanted effects) by using find . -type f

The grep ASCII is both too loose and too tight. It is too loose in that it will pass files containing ASCII in the name as well as in the type that file reports. It is too tight in that there are many text file types which spoovy probably wants to cat that do not have ASCII in the type that file reports.

The cut -d ":" -f1 will not do as intended for files with names including the : character.

The cat ` ` will not work on files with names including embedded whitespace (space, tab, newline -- only the first is likely) in their names.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
List all files in a directory recursively, without the directories? b10m3ch4 Linux - General 4 11-12-2009 06:07 PM
How to list all files recursively, in a non-broken list? nyle Linux - Newbie 1 12-16-2008 11:52 PM
recursively list files and dirs based on permission sneakyimp Linux - Software 2 10-16-2007 07:37 PM
list recursively files with for xeon123 Programming 6 04-04-2007 04:38 PM
List all files and recursively open directories. unreal128 Linux - General 2 07-16-2005 03:06 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 07:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration