[SOLVED] Bash/awk extract function from file.

grail · 08-02-2013, 11:55 AM

So clearly no need to have this in a single line entry as who could ever want to type it out all the time.
So in a script it looks a little cleaner:

Code:

#!/usr/bin/awk -f

{
	line=$0
	gsub(/"[^"]*"/, "", line)
	gsub(/'[^']*'/, "", line)
	gsub(/#.*$/, "", line)
}

/test()|function test/ {

	if( sub("test()|function test", "", line) ) {
		start=1
		brackets=0
	}
}

/[{}]/ && start{

	brackets+=split(line,_,"{") - 1
	brackets-=split(line,_,"}") - 1
}

start{
	print $0" - "brackets
	
	if (brackets <= 0){
		print "EXITING"
		exit
	}
}

You can shift the "EXITING" back if you prefer, but I think the rest is a little cleaner.

apottere · 08-02-2013, 03:26 PM

Yeah, I think I might give up on this one. The intent was to have "modules" of bash files, and be able to extract certain functions from them to insert into others. I can get the same kind of functionality with "@start_test <function test....> @end_test" with much less hassle.

konsolebox · 08-02-2013, 03:39 PM

Quote:

Originally Posted by apottere

The intent was to have "modules" of bash files, and be able to extract certain functions from them to insert into others.

Oh you might as well consider this one for your solution and an example project that makes use of it, by the same author.

David the H. · 08-02-2013, 03:45 PM

I doubt you'll ever be able to write a completely reliable function extractor using regex-based tools like awk. The nested brace patterns are just too unpredictable. What if there are brace characters sitting in comments, for example?

Actually, I have a suggestion for a much easier way to get what you want (maybe):

Code:

( . test.sh && declare -f test )

This will print out the test function, as bash sees it. It will however be restructured into what declare considers reusable script syntax, so the original formatting and things like comments will disappear.

David the H. · 08-02-2013, 04:01 PM

I just saw the updated posts that appeared after I started writing, so here's another idea.

If you want to have a kind of library of functions, and the input files are completely under your control, then just separate each function with a fixed delimiter string, and use that to have awk extract the ones you want.

function_file.txt:

Code:

###---###
test_a(){
    echo "This is test function A"
}
###---###
test_b(){
    echo "This is test function B"
}
###---###

Code:

$ awk 'BEGIN{ RS="\n###---###\n" } /^test_b/' function_file.txt
test_b(){
    echo "This is test function B"
}

apottere · 08-02-2013, 04:29 PM

Quote:

Originally Posted by David the H.

I doubt you'll ever be able to write a completely reliable function extractor using regex-based tools like awk. The nested brace patterns are just too unpredictable. What if there are brace characters sitting in comments, for example?

Actually, I have a suggestion for a much easier way to get what you want (maybe):

Code:

( . test.sh && declare -f test )

This will print out the test function, as bash sees it. It will however be restructured into what declare considers reusable script syntax, so the original formatting and things like comments will disappear.

This is exactly what I was looking for, I don't much care about formatting when the executable is created. In fact, is there any way to compress it further with declare? The man pages didn't seem to have anything like that.

apottere · 08-02-2013, 04:31 PM

Quote:

Originally Posted by David the H.

I just saw the updated posts that appeared after I started writing, so here's another idea.

If you want to have a kind of library of functions, and the input files are completely under your control, then just separate each function with a fixed delimiter string, and use that to have awk extract the ones you want.

function_file.txt:

Code:

###---###
test_a(){
    echo "This is test function A"
}
###---###
test_b(){
    echo "This is test function B"
}
###---###

Code:

$ awk 'BEGIN{ RS="\n###---###\n" } /^test_b/' function_file.txt
test_b(){
    echo "This is test function B"
}

This is a much cleaner version of what I had (and I may steal it for future use), but declare -f is what I was looking for. Thanks for the help!

apottere · 08-02-2013, 04:35 PM

Quote:

Originally Posted by konsolebox

Oh you might as well consider this one for your solution and an example project that makes use of it, by the same author.

This is a cool concept too, but I'm not sure I want to link at run-time.

konsolebox · 08-02-2013, 04:45 PM

Using markers is actually another good idea. I would also suggest using function-start and function-end markers to make it more specific to functions only even if there are some other lines around.

Code:

###-begin-###
test_a(){
    echo "This is test function A"
}
###-end-###

konsolebox · 08-02-2013, 04:51 PM

Quote:

Originally Posted by apottere

This is a cool concept too, but I'm not sure I want to link at run-time.

Just curious. Why did you have to extract the functions for?

And what kind of modules? I mean how would you use them?

apottere · 08-02-2013, 04:57 PM

I'm trying to modularize my code, so I can write a script with modules and then link it all together. This means I only need the modules present once, and after that I have a working executable that I can use anywhere.

David the H. · 08-02-2013, 05:16 PM

I'm not exactly sure what you mean by "compress it further", but no, there's no way that I know of to control the format of the declare output. It's all handled internally.

konsolebox · 08-02-2013, 05:37 PM

Quote:

Originally Posted by apottere

I'm trying to modularize my code, so I can write a script with modules and then link it all together. This means I only need the modules present once, and after that I have a working executable that I can use anywhere.

Then Shell Script Loader could be one solution. Scripts that work with Shell Script Loader could also be compiled to be made as one file. See the section about external compilers. And PlayShell makes use of all those features. You could examine it to see how it's done. See start.sh, compile.sh and loader/*.

It's also helpful since you can test your code with your modules separated without needing to compile all together first and makes debugging easier to detect which file actually had the error.

grail · 08-02-2013, 11:15 PM

Personally I would just place all your functions / modules in a group of files and then simply source the appropriate file so the function / module is now available.

apottere · 08-02-2013, 11:31 PM

Quote:

Originally Posted by grail

Personally I would just place all your functions / modules in a group of files and then simply source the appropriate file so the function / module is now available.

The object is to end up with a completely contained script I can move anywhere and have work, not one that relies on other files being in the correct place. This is normally a fine solution, just not in this case.