Extracting text at a dynamic location
 

Consider a file that is structured like this

I'd like to write a script that :


I'm curious if there's a simple sed/awk solution to this. If not, I will turn to python.
But simpler is better.

 

This sounds familiar... https://www.linuxquestions.org/questions/programming-9/awk-sed-bash-script-to-display-last-changelog-entry-4175716978

That's a very similar problem, so yes, Awk can do this too, and you should be able to apply what you've learned from that thread.

Here are some hints: Use "\n\* " as a record separator, "\n\*\* " as a field separator, and "$NF" to refer to the last field in a record.

Using that information have a go yourself and if you get stuck show your efforts.

Thanks boughonp! A little update, I need to access last line from a field that has multiple lines,
like so :

In this example, I need to extract last line which is :

 

There's a few ways to access the last line of a variable, perhaps the simplest is to split on newlines and access the last element of the resulting array.

Awk's split is slightly differently to other languages:
Where "input_string" is changed to whatever variable/expression contains the lines, and "my_array" and "my_array_len" are variable which can be named however you like.

(If there's a trailing newline in the input, a -1 could be added to counter that.)

Or pipe it to tail -1

Thank you boughtonp! that code will save me a gratuitious call to length() as in my code

@MadeInGermany it's a shame to use another subprocess when you can do everything inside awk.

The code is now in its own file



The problem is that task variable contains special characters "[" "]", so the if ($i ~ task) condition will never meet.
With the other test if ($i ~ "learning to use getopts") I get a match :

But with the if ($i ~ task) code I get no match

 

To do a non-regex find, use index(haystack,needle) - returns position of match, with a starting string returning 1.

But it can be useful to convert a string to a regex pattern, by adding backslashes where required:
Then the pattern can have additional regex metacharacters prefixed/suffixed as required (e.g. to ensure start/end of string, variable prefixes, etc.)

gsub didn't work


but index did.

 

The "text" bit was intended to be generic - in your context you'd want something like:

But this is approach is more for if you want to add to the pattern - if index is sufficient that's the simpler and more efficient approach.

oops, you were right ^^', didn't check the name of the variable.

But there's something intruiguing, if I use the original variable tasks I get a very big load of backslashes printed out (see https://i.imgur.com/mgoKUtJ.png).

If I use a copy of the variable, it works as exepcted

This is handy. Should I turn into a function? I might use it in future scripts.

Interesting. I guess there's a quirk related to how it modifies the variable in place, and the act of assignment changes some property to resolve that.

Unless there's some documented reason for it, you should check which implementation + version of Awk you're using and probably raise it as a bug.

edit: I was distracted earlier - it's because the gsub is occurring inside a loop, and each iteration adds/doubles backslashes. If used, it should be done prior to the loop.


Yep - it was translated from one of the functions in a regex library for a different language.

I was a little surprised it wasn't already a built-in function.

Oh my dear :eek:
Go for the index() function!

Yep, the benefit of escaping is for use inside a larger pattern; when not doing that, the index function is simpler, clearer, more efficient, etc.


Also, I wasn't paying attention earlier - the excess slashes are due to the replace being performed inside a loop; if used it needs to only be done once (hence why resetting the variable immediately prior hid the issue).