Graphing csv data
 

Hi all,

I have a little data logger running an embedded version of Linux. It logs energy data from my solar inverters and saves them in a gzipped .csv file. Previously, those files were pushed out to a third party server which would display them in a nice graphed dashboard to be used as a kiosk. That company is no longer in business so what I would like to do is process that raw data myself. I am comfortable in a Linux environment but by no means an expert.

What is the best way to do this?
I thought about pushing that data into a database and then using something like grafana to visualize it? But that seems a little over my head.

Any ideas are appreciated.

Perhaps take a look at gnuplot?

I used R (https://www.r-project.org/) for that kind of tasks. You can read csv into data frame and then operate on it. For large csv files use data.table package and for advanced graphing ggplot2. The spectrum of available diagrams is quite wide and they can be saved as jpg, png, pdf etc. You can automate the whole task and write Rscript.

RRDTool and rrdgraph perhaps?

+1 for this. VERY useful for generating graphs, without a huge database and a ton of overhead.

Thanks for all your suggestions. So far I have the data being consolidated, formatted, and then loaded into a MySQL database. Just need to figure out how to graph it now. I've tried out your recommendation and have also played around with pandas a little, but still looking for that perfect solution.

How would I import my data(.csv or .txt) in to the rrd? And I would need to convert all my timestamps in to seconds since 1970 correct?