Script need to choose plain text documents

ChartmanSg · 12-04-2008, 10:14 AM

Hi there,

I've written a bash script to process plain text documents.

Unfortunately, the files that I get daily include other formats like swf, PNG, JPEG, HTML, XML, unknown, C Source Code, etc.

What script do I need to select just the plain text documents and ignore the rest?

Would appreciate any help from the experts.

James

indienick · 12-04-2008, 10:20 AM

You can use the "file" command to get details about a file - then, using the output from that, pipe it to a sandbox to where your script can process the plain-text files.

zer0x333 · 12-04-2008, 10:21 AM

Could you use the 'file' command to identify the filetype and act accordingly?

ChartmanSg · 12-04-2008, 10:25 AM

Thanks for the replies guys.

I've identified the files that I need as plain text documents.

But I dont know and cant seem to find the file extensions for plain text documents.

A normal text file would be file.txt

But what extension does a plain text document have?

James

zer0x333 · 12-04-2008, 10:30 AM

I would ignore the file extensions if at all possible, and use the output from 'file'.

Plain ASCII text files are described as 'ASCII text'.

indienick · 12-04-2008, 10:33 AM

Please bear in mind that file extensions are really only used for organization, and are a left-over from the days of DOS.

A plain-text document, under the DOS ideal, would also have a ".txt" extension - BUT - not everyone likes to use extensions on their files (especially plain-text). So, use "file" command for a file you know is plain-text, and run it on a second file that is something completely different (C-source code, or something). Note the differences in output between the two files, and construct an algorithm something like this:

Code:

1. Use the "file" command on file X; pipe the output to "grep -i".
2. If grep returns a result, rename the file to have a ".txt" extension.
3. If grep does not return a result, then move on to the next file in the glob and GoTo Step 1.

So, perhaps something like:

Code:

$ file fileX | grep -i "ascii text" && mv fileX fileX.txt && plain-text-thingamajig-script fileX.txt

ChartmanSg · 12-04-2008, 10:38 AM

Thank you, indienick and zer0x333 !!!

I finally see light at the end of my tunnel.

Now I know what to do

You're both godsends.

James

indienick · 12-04-2008, 10:39 AM

*blushes* Well, now...I wouldn't go that far, but you're very welcome!

zer0x333 · 12-04-2008, 10:44 AM

No problem, not as helpful as indienick! Slack xD

Good Luck!