[SOLVED] Want to grep data between square brackets(Including brackets)

harshagr · 02-12-2018, 10:28 AM

I am trying to awk data from a curl output.

My requirement is to grep data between square brackets [ ].

Eg: In a file I have a output as {"command1":{"cmd":"get_msp_audit_export","response":[{"eventdate":"2018-04-06","facility_fee":null]},"header":{"src_sys_name":"sharedservices","ver":"1.0","archtics_version":"v999","ats_version":"4.4 2.0","src_sys_type":"2"}} and so on, from this I want grep the output as [{"eventdate":"2018-04-06","facility_fee":null] only(Including square brackets).

The occurrence of square brackets is only one.

I tried the below command but removes the square brackets too. Please help ASAP.

#awk 'NR>1{print $1}' RS=[ FS=]

BW-userx · 02-12-2018, 10:42 AM

Code:

$ ls ~/scripts/*  | egrep "[ ]"
my [ bracket ] file

filename is actually 'my [ bracket ] file'

ohh you want to remove [ ] and leave the rest using grep?

Code:

$ ls ~/scripts/*  | awk ' { gsub ( /[][]/, "" ) ;  print }'
my  bracket  file

your example

Code:

$ string={"command1":{"cmd":"get_msp_audit_export","response":[{"eventdate":"2018-04-06","facility_fee":null]},"header":{"src_sys_name":"sharedservices","ver":"1.0","archtics_version":"v999","ats_version":"4.4 2.0","src_sys_type":"2"}}

$ string2=$(echo -e "$string" | awk '{ gsub ( /[][]/, "") ; print}')
 
$ echo $string2
{command1:{cmd:get_msp_audit_export,response:{eventdate:2018-04-06,facility_fee:null},header:{src_sys_name:sharedservices,ver:1.0,archtics_version:v999,ats_version:4.4 2.0,src_sys_type:2}}

your [ ] are gone.

harshagr · 02-12-2018, 10:50 AM

Does not work. It has to be something with awk to work.

harshagr · 02-12-2018, 11:03 AM

Quote:

Originally Posted by BW-userx

Code:

$ ls ~/scripts/*  | egrep "[ ]"
my [ bracket ] file

filename is actually 'my [ bracket ] file'

ohh you want to remove [ ] and leave the rest using grep?

Code:

$ ls ~/scripts/*  | awk ' { gsub ( /[][]/, "" ) ;  print }'
my  bracket  file

your example

Code:

$ string={"command1":{"cmd":"get_msp_audit_export","response":[{"eventdate":"2018-04-06","facility_fee":null]},"header":{"src_sys_name":"sharedservices","ver":"1.0","archtics_version":"v999","ats_version":"4.4 2.0","src_sys_type":"2"}}

$ string2=$(echo -e "$string" | awk '{ gsub ( /[][]/, "") ; print}')
 
$ echo $string2
{command1:{cmd:get_msp_audit_export,response:{eventdate:2018-04-06,facility_fee:null},header:{src_sys_name:sharedservices,ver:1.0,archtics_version:v999,ats_version:4.4 2.0,src_sys_type:2}}

your [ ] are gone.

No, Actually I want the data between square brackets only(output should have square brackets too), starting from [ and ending with ]. For my eg output should come as [{"eventdate":"2018-04-06","facility_fee":null]

norobro · 02-12-2018, 11:30 AM

See the match function: https://www.gnu.org/software/gawk/ma...ring-Functions

Or using your awk script, you could explicitly print out the brackets - print "["$1"]"

harshagr · 02-12-2018, 11:41 AM

Quote:

Originally Posted by norobro

See the match function: https://www.gnu.org/software/gawk/ma...ring-Functions

Or using your awk script, you could explicitly print out the brackets - print "["$1"]"

I tried all but it does not work, Can you please help with a exact command to get the desired output I am looking for.

norobro · 02-12-2018, 11:46 AM

So this does not work?

Code:

awk 'NR>1{print "["$1"]"}' RS=[ FS=]

Post the commands that you tried.

harshagr · 02-12-2018, 11:53 AM

Quote:

Originally Posted by norobro

So this does not work?

Code:

awk 'NR>1{print "["$1"]"}' RS=[ FS=]

Post the commands that you tried.

I Tried awk '$0=="[" {p=1}; p; $0=="]" {p=0}' and awk 'NR>1{print $1}' RS=[ FS=]

First command prints a blank output and second command prints the output as {"eventdate":"2018-04-06","facility_fee":null. But I want the output as [{"eventdate":"2018-04-06","facility_fee":null]

BW-userx · 02-12-2018, 12:03 PM

Code:

userx@slackwhere:~
$ echo -e $string | awk {'print "["$1"]"'}
[{command1:{cmd:get_msp_audit_export,response:[{eventdate:2018-04-06,facility_fee:null]},header:{src_sys_name:sharedservices,ver:1.0,archtics_version:v999,ats_version:4.4]

if you want to remove every insistence of [something] and remove everything else, but if you got multiple
[something][something][something]

Code:

 
$ echo -e $string  | sed 's/.*\[\([^]]*\)\].*/\1/g'
{eventdate:2018-04-06,facility_fee:null

#gets the first instance of [ something ] .. 
#you'll just need to put the [ ] back 

$ string={"command1":{"cmd":"get_msp_audit_export","response":[{"eventdate":"2018-04-06","facility_fee":null]},"header":{"src_sys_name":"sharedservices","ver":"1.0","archtics_version":"v999","ats_version":"4.4 2.0","src_sys_type":"2"}}

figure out how to work down your string to get the next insistence of [ something ]

danielbmartin · 02-12-2018, 03:28 PM

I obtained good results with all of these ...

Code:

echo
echo; echo "Method #1 of LQ member danielbmartin."
awk 'BEGIN{FS="[][]"} {print "["$2"]"}' $InFile >$OutFile
echo "OutFile ..."; cat $OutFile; echo "End Of OutFile ($(wc -l <$OutFile) lines)"

echo
echo; echo "Method #2 of LQ member danielbmartin."
awk -F"[][]" '{print "["$2"]"}' $InFile >$OutFile
echo "OutFile ..."; cat $OutFile; echo "End Of OutFile ($(wc -l <$OutFile) lines)"

echo
echo; echo "Method #3 of LQ member danielbmartin."
 cut -d[ -f2 $InFile   \
|cut -d] -f1           \
|sed 's/\(.*\)/[\1]/'  \
>$OutFile
echo "OutFile ..."; cat $OutFile; echo "End Of OutFile ($(wc -l <$OutFile) lines)"

echo
echo; echo "Method #4 of LQ member danielbmartin."
sed 's/\(.*\[\)\(.*\]\)\(.*$\)/[\2/' $InFile >$OutFile
echo "OutFile ..."; cat $OutFile; echo "End Of OutFile ($(wc -l <$OutFile) lines)"

Daniel B. Martin

.

norobro · 02-12-2018, 07:24 PM

I assume the homework deadline has passed so here is a version with the awk match function:

Code:

awk '{ match($0, /(\[.*\])/, capture)} {print capture[1]}' input_file

@danielbmartin - Your awk scripts do work but I'm having trouble understanding how -F"[][]" is evaluated. The manpage says:

Quote:

If FS is a single character, fields are separated by that character. If FS is the null string, then each individual character becomes a separate field. Otherwise, FS is expected to be a full regular expression.

keefaz · 02-12-2018, 08:17 PM

Code:

egrep -o '\[[^]]*]' file

danielbmartin · 02-12-2018, 09:09 PM

Quote:

Originally Posted by keefaz

Code:

egrep -o '\[[^]]*]' file

Ding ding ding ding ding! We have a winner! keefaz gets the Blue Ribbon for brevity and elegance!!

Daniel B. Martin

.

danielbmartin · 02-12-2018, 09:39 PM

Quote:

Originally Posted by norobro

@danielbmartin - Your awk scripts do work but I'm having trouble understanding how -F"[][]" is evaluated.

I hadn't read the manpage. Now that I have, I don't understand it.

This awk ...

Code:

awk -F"[dh]" '{print $2}' <<<"abcdefghi"

... says there are two Field Separator characters, d and h.
The result is efg

This awk ...

Code:

awk -F"[][]" '{print $2}' <<<"abc[efg]i"

... says there are two Field Separator characters, ] and [.
The result is efg

There is one "wrinkle" to note. Coding this: -F"[[]]" doesn't work because it looks like the null string, so you reverse the Field Separator brackets and code -F"[][]" instead.

Daniel B. Martin

.

harshagr · 02-13-2018, 03:47 AM

Quote:

Originally Posted by norobro

So this does not work?

Code:

awk 'NR>1{print "["$1"]"}' RS=[ FS=]

Post the commands that you tried.

It worked for me. Thanks a Ton.