[SOLVED] how to list all ascii/text files in a folder recursively?

suhasingale · 11-18-2011, 12:58 AM

Hi Geeks,

I want to create a list of ascii/text/xml files for one folder recursively. I know I can use the find command to list all the files and file command to check the file type. But I am not sure how to combine both and create a list of only text/ascii/xml files excluding the data, binary etc types.

Please guide me through.

Regards,
Suhas

davemguru · 11-18-2011, 01:50 AM

Quote:

Originally Posted by suhasingale

Hi Geeks,

I want to create a list of ascii/text/xml files for one folder recursively. I know I can use the find command to list all the files and file command to check the file type. But I am not sure how to combine both and create a list of only text/ascii/xml files excluding the data, binary etc types.

Please guide me through.

Regards,
Suhas

Simple search - all files in the current directory NO recursion.

Code:

file *|grep text|grep -v OpenDocument

lists all ascii/text/xml/html/shell script/ascii english text (like ".csv" files) on STDOUT.
The "-v" option to grep is saying "not these". So, for example - if you don't want the english text or the UTF-8 ascii files you could add some more pipes and exclusions.

You know that "find" will recurse from wherever you tell it. So, you could use find to create a temporary list (in /tmp for example) and then iterate through that list in a simple shell script -

Code:

while read pathname
do
..... file $pathname |blah| blah| grep |blah
done </tmp/whatever

Said shell script could then be given a name e.g. - homework.sh and then you could execute the script and redirect it's standard output to a file which would then contain your results.

Code:

 ./homework.sh >results

BTW - Some people view "Geek" as a derogatory term. Perhaps it would be better to address your requests "Hi helpful people". Or if you felt that too presumptuous - you could put "possibly" in parenthesis prior to the word "helpful".

dave

suhasingale · 11-18-2011, 05:00 AM

Thanks for the solutions davemguru.

I wrote a simple script myself as below:

Quote:

#!/bin/ksh

find <folder/path> -type f > /tmp/1
temp="/tmp/1"
echo > /tmp/2
for i in `cat $temp`
do
temp1=`file $i | egrep -i "text|xml"`
if [[ $temp1 ]]
then
echo $i >> /tmp/2
fi
done

David the H. · 11-18-2011, 01:28 PM

(The links I'm posting are for bash, but most of what they say applies to ksh too.)

1) Please use [code][/code] tags around your code, to preserve formatting and to improve readability. Do NOT use quote tags, as they don't protect whitespace.

2) Don't read lines with for. Also, cat is usually unnecessary for reading from single files in modern shells.

3) $(..) is highly recommended over `..`

4) file is, in my experience, notoriously unreliable when it comes to detecting text files. It often mistakenly detects simple text as being, say, a lisp script. It has something to do with the way it parses the beginning of the file to determine it's "magic" type. At the very least try using the -i option and parse the mime-type info instead.

5) Instead of using grep, try generating a simple list of file names, then loop through the list, testing the output of file for each one directly. Perhaps something like this:

Code:

while read file ; do
	if [[ $( file -i "$file" ) =~ plain ]]; then
		echo "$file is a plain text file"
	fi
done <filelist

P.S. I don't think you'll find many people here who see "geek" as being derogatory. Indeed, "geek pride" is on the rise, as subjects like gaming and comics that were once nerdy are now mainstream, and it's becoming clearer to everyone that those who really know the tech are the ones who hold the reins.