Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have an input file (10000 lines) named ‘out’ in the given format. Each two lines represent one structure. I am giving this data file as input to a shell program which creates 5000 folders; one directory for each chromosomes using built-in tool ‘mfold SEQ=input’ command. The shell program is ‘process.awk’ given below. I am executing with a command ‘awk –f process.awk out’. The program executed and created 1020 directories and after that it is giving error like this:
awk: process.awk:6: (FILENAME=out FNR=1022) fatal: can't redirect to `dir1021/input' (No such file or directory)
I am not able to correct this. Help for this highly appreciated. Thanks in advance.
I'm curious if awk is running out of file descriptors (too many open files) since there are no close() calls in the script. We don't know what OS this is running on (do we?); GNU gawk apparently has no limits (within reason?) but other awk implementations may have limits.
I am working on Linux OS. How to modify the above awk program? help me.
Quote:
Originally Posted by GrapefruiTgirl
I'm curious if awk is running out of file descriptors (too many open files) since there are no close() calls in the script. We don't know what OS this is running on (do we?); GNU gawk apparently has no limits (within reason?) but other awk implementations may have limits.
Have you made progress, or done further investigation yet? If so, what did you find?
I am not sure if what I suggest above is correct; but the way I figure it, your script is trying to create fd #1021, which would be the 1022nd fd created based upon the input data; the `mfold` system command's fd would account for the 1023rd fd; the input file would account for the 1024th fd; the awk script itself *might* account for the 1025th fd, or maybe each of your `mfold` commands accounts for its own fd -- either way makes for a total of 1024 open fd's and an attempt being made to open another one, which fails. The failure hints at a limit of 1024 open fd's for your awk version.
If this is all correct (again, I do not know if it is), then the solution would be to issue the close() command once for every open file descriptor that gets created, after the script is finished using that descriptor. So if it were me, I would use close("fd name here") after every `print` command you use.
Still I am in investigation round the clock. Pls modify the given script by adding close statements madam. I will try that also
Thanks in advance.
Quote:
Originally Posted by GrapefruiTgirl
Have you made progress, or done further investigation yet? If so, what did you find?
I am not sure if what I suggest above is correct; but the way I figure it, your script is trying to create fd #1021, which would be the 1022nd fd created based upon the input data; the `mfold` system command's fd would account for the 1023rd fd; the input file would account for the 1024th fd; the awk script itself *might* account for the 1025th fd, or maybe each of your `mfold` commands accounts for its own fd -- either way makes for a total of 1024 open fd's and an attempt being made to open another one, which fails. The failure hints at a limit of 1024 open fd's for your awk version.
If this is all correct (again, I do not know if it is), then the solution would be to issue the close() command once for every open file descriptor that gets created, after the script is finished using that descriptor. So if it were me, I would use close("fd name here") after every `print` command you use.
Still I am in investigation round the clock. Pls modify the given script by adding close statements madam. I will try that also
Thanks in advance.
With the information I have given thus far, plus the link given above, and based the snippets of code you have posted in the past, I am reasonably confident that you yourself are capable of adding a single close() statement with the right stuff inside the brackets, after the print statement. I have used numerous close() statements in my awk code over here: http://www.linuxquestions.org/questi...2/#post4126234
Have a look and see what I did. Modify your code. Test it. Show us the results if it fails (copy + paste the errors of execution of your program) and show us the code again, with the close() statements added.
Please allow me to remind you again though: I do not know if this will address the issue, or if the number of fd's is even the problem, but if it were me, I would be doing exactly what I'm suggesting you try: close() statements.
As I have said in other posts from same OP on same topic, why not just use bash seeing all the calls to system?
A simple while loop could easily read from the file and then issue all your command as you have inside the system calls.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.