How to list contents of all files recursively

spoovy · 01-22-2011, 04:08 PM

What I would like to do is to print the contents of all text files in a particular directory, recursively. Problem being that there are directories and possibly binaries scattered around in the filesystem as well.

Trying cat * works as long as there are no directories in there, but when there are it gives an error instead and prints nothing.

I'm sure it's easy using file -f or something but I can't figure it!

Any ideas?

Thanks in advance

spoov

AlucardZero · 01-22-2011, 04:32 PM

Code:

find /dir -type f -exec cat '{}' +;

?

catkin · 01-23-2011, 10:26 AM

catting some non-text files is likely to do strange things to the terminal; it would be prudent to run the file command on each file found and only cat the file if it is a text type. This could most easily be done by writing a script and calling that from find (beware that AlucardZero's +; has been changed to \; so the script is called with a single file name):

Code:

find /dir -type f -exec /path_to_your_script '{}' \;

The script could be something like this (not tested):

Code:

#!/bin/bash

case "$(file "$1" | sed 's/.*: //' ) in
    *'ASCII C'*text* \
        | *'HTML document text' \
        | *'ISO-8859'*text* \
        | *'M3U playlist text' \
        | *'UTF-8 Unicode'*text* \
        | *'Windows Registry text'* \
        | 'XML document text' \
        | *'a /bin/bash script text executable' \
        | *'exported SGML document text' \
            )
                cat "$1"
                ;;
    * )
        echo "Skipped (not a recognised text type): $1"
esac

It could be made more robust and could loop over multiple file names given as arguments but is probably "good enough" as long as you don't have a lot of files.

spoovy · 01-23-2011, 02:30 PM

Quote:

Originally Posted by AlucardZero

Code:

find /dir -type f -exec cat '{}' +;

?

Excellent! Thanks v much, that worked for my purpose.

@Catkin - you're right about cat-ing non-text files possibly doing strange things, it buggered up my console on one occasion, causing the font to change to some non-ASCII gobbldigook.
Thanks very much for your additional info, i'll try that out.

colucix · 01-23-2011, 02:43 PM

Quote:

Originally Posted by spoovy

@Catkin - you're right about cat-ing non-text files possibly doing strange things, it buggered up my console on one occasion, causing the font to change to some non-ASCII gobbldigook.

In that case you can type reset, even if you cannot read what you type, press enter and the terminal restores its normal status.

spoovy · 01-23-2011, 03:10 PM

Catkin - What do you mean by "(beware that AlucardZero's +; has been changed to \; so the script is called with a single file name):"?

I don't know what the + character does in the first place tbh!

AlucardZero · 01-23-2011, 03:27 PM

man find -

Code:

      -exec command ;
              Execute command; true if 0 status is returned.  All following arguments to find are
              taken to be arguments to the command until an argument consisting of ‘;’ is encoun‐
              tered.  [..cut..]

       -exec command {} +
              This  variant of the -exec action runs the specified command on the selected files,
              but the command line is built by appending each selected file name at the end;  the
              total  number  of  invocations  of the command will be much less than the number of
              matched files. [..etc..]

It's the difference between

Code:

cat a
cat b
cat c

and

Code:

cat a b c

Manjunath1847 · 01-24-2011, 05:58 AM

cat `find . -name "*" | xargs file | grep ASCII | cut -d ":" -f1`

Quote:

Originally Posted by spoovy

What I would like to do is to print the contents of all text files in a particular directory, recursively. Problem being that there are directories and possibly binaries scattered around in the filesystem as well.

Trying cat * works as long as there are no directories in there, but when there are it gives an error instead and prints nothing.

I'm sure it's easy using file -f or something but I can't figure it!

Any ideas?

Thanks in advance

spoov

catkin · 01-24-2011, 11:19 AM

Quote:

Originally Posted by Manjunath1847

cat `find . -name "*" | xargs file | grep ASCII | cut -d ":" -f1`

A good start but a few issues with that.

In find . -name "*", -name "*" gives the default behaviour so could as well be find . And spoovy has said only ordinary files are relevant -- not directories or any of the special file types -- so we can cut down on the files passed down the pipeline (some of which, in perverse cases, could cause unwanted effects) by using find . -type f

The grep ASCII is both too loose and too tight. It is too loose in that it will pass files containing ASCII in the name as well as in the type that file reports. It is too tight in that there are many text file types which spoovy probably wants to cat that do not have ASCII in the type that file reports.

The cut -d ":" -f1 will not do as intended for files with names including the : character.

The cat ` ` will not work on files with names including embedded whitespace (space, tab, newline -- only the first is likely) in their names.