How to ask SED to remove ALL but PATTERN?

mprowe · 03-22-2024, 04:50 AM

Hi,
I wish to build a csv file from the information in my MQTT server.
This is my command:

Code:

~> mosquitto_sub -h 192.168.xxx.yyy -t topic/sub1/sub2/sub3 -u myuser -P mypass > file.txt

This is fine but the output contains more data than I need:

Code:

{"electricitymeter":{"timestamp":"2024-03-22T07:44:58Z","energy":{"export":{"cumulative":0.000,"units":"kWh"}... truncated for brevity!

Can anyone suggest a sed command that I can pipe through, that will remove all but two patterns:
pattern 1: "timestamp":"2024-03-22T07:44:58Z",
pattern 2: "unitrate":0.15190,
Note that "timestamp" & "unitrate" are unique and fixed but the values (between the colon ':' and comma ',' are variable.

Best regards, Martin

pan64 · 03-28-2024, 02:23 AM

this is a json file and you ought to use a json parser to gather the required information instead of sed/grep/awk.
You can use for example perl, python or jq.

syg00 · 03-28-2024, 02:45 AM

Quote:

Originally Posted by mprowe

Can anyone suggest a sed command that I can pipe through, that will remove all but two patterns:

Multitudes likely.
We are not here to do your bidding - make an effort, and explain where you have issues. Someone may be able to help.

The above is the canonical answer, but in this case, if that is representative data, sed (with regex) can handle it reasonably straight-forwardly.

Turbocapitalist · 03-28-2024, 02:57 AM

I second the recommendation to use perl, python or jq. Furthermore, use a pipe rather than a redirection so that the processing script can work on the data as it arrives. Though Perl and Python can subscribe to MQTT directly so a pipe would only be needed for jq.

lvm_ · 03-28-2024, 03:30 AM

If your sed fixation is strong you may use \1 subexpression references to leave only what you want - see https://www.gnu.org/software/sed/man...pressions.html But yes, there are better tools for that.

MadeInGermany · 03-28-2024, 06:05 AM

My idea with grep -o

Code:

grep -Eo -e '"timestamp":"[^"]*"' -e '"unitrate":([-+0-9.Ee]+|"[^"]*")'