how to automate file processing in bash?
Ok
The heading might not have made sense.. Here is what I need to do There is a perl script which takes input of a .txt file and outputs the result to STDOUT!!! So in essence we run it as on bash prompt Code:
./script.pl INPUTFILE1.txt > INPUTFILE1_RESULT.txt How do I do this?? Also to take it to next level.. lets say there are multiple folders which have the INPUTFILE.txt.. it possible to do this same thing as above recursively and create a new output folder (if does not exist) as well for each folder processed with _RESULT in end name of folder.. ofcourse don't want to complicate things .. I can manually go into them and point the result to a manually created folder.. Again solution to 1st problem itself will be great if it makes any difference, the code will run in RHEL.. I cannot modify the .pl script.. so just want to modify the execution style on bash Cheers |
NO bash expert here, so fix the syntax:
for i in /wherever do; ./script.pl $i > $i.results; # probably need some brackets around '$i' next i; |
Code:
for f in `cat inputfilelist` |
Would not the best option be to simply extend the perl script to cope with the new requirement?
|
Code:
#!/bin/sh |
oh, no! see bash variable substitution:
Code:
fileext=${FILE##*.} |
Using Bash:
Code:
find DIR(s)... -name '*.txt' -print0 | while read -rd "" source ; do The read -rd "" uses the Bash read built-in to read them into variable source one by one in the while .. ; do ... done loop. target gets set to the value of source but with everything after the final . replaced with _RESULT.txt. If you want to skip any existing result files, use Code:
[[ -e "$target" ]] || ./script.pl "$source" > "$target" |
Quote:
|
Quote:
Nominal Animal demonstrates the power of find very nicely, and also demonstrates and explains a well structured way to approach the problem: iteration over a file set, especially with recursion, suggests using find. The output of find is a list, which feeds the while loop, so there is the iteration part of the problem solved. Once that part of the problem is addressed, he assembles a couple of key variables that are the arguments to the perl script. Finally, that last thing in each iteration is to invoke the perl script with arguments that are variables with nice human-readable names. There is more to learn from Nominal's code than just how to solve the problem. It is a good demonstration of how structure the solution. --- rod. |
All times are GMT -5. The time now is 01:00 AM. |