LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-09-2018, 10:36 PM   #1
rakhighosh
LQ Newbie
 
Registered: Jan 2018
Posts: 1

Rep: Reputation: Disabled
calculation of average from a csv file using shell script


I have a csv file having values of four different parameters . The values of the parameters for a single day are many and I want to find the average of the values using a shell script.
My csv file looks like this

year month date time rainfall(cm)
2012 12 5 10.00 12
2012 12 5 12.00 14
2013 04 3 10.00 16
2013 04 3 12.00 17
I want to calculate the average of rainfall for a single day.How to write a shell script for this?

Last edited by rakhighosh; 01-09-2018 at 10:38 PM. Reason: the values were not properly assigned
 
Old 01-09-2018, 11:45 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191
Welcome to LQ

Please show what you have attempted so far?
 
Old 01-10-2018, 12:22 AM   #3
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=15, FreeBSD_12{.0|.1}
Posts: 6,264
Blog Entries: 24

Rep: Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194Reputation: 4194
Agreed it would be nice to show your efforts to this point.

I think a simple awk script would be an easy solution which would also provide decimal average values which a shell script will not do.

By way of encouragement I wrote a quick and dirty awk which does the following pseudo-code (assumes dates are not interleaved, prints average for each date):

Code:
BEGIN { 
       #...initialize date match, count and total vars 
}
Same date{
       #Increment count, add amount to total
       next
}
Count > 0{
       #Change of date so print last date and average, reset date match, count and total vars
       next
}
{
       #First line, set initial date match, increment count, set total to amount
}
END{ 
       #Last line, print last date and average 
}
When run with your example data it produces this...

Code:
awk -f rain.awk rain_data
Average: 2012 12 5 13
Average: 2013 04 3 16.5
I leave the actual code as an exercise for you! (Pretty simple!)

Last edited by astrogeek; 01-10-2018 at 12:30 AM. Reason: Grammar
 
Old 01-10-2018, 01:09 AM   #4
Sefyir
Member
 
Registered: Mar 2015
Distribution: Linux Mint
Posts: 634

Rep: Reputation: 316Reputation: 316Reputation: 316Reputation: 316
If interested in interpreting large amounts of data, you can try using pandas with python
https://pandas.pydata.org/

Code:
import pandas as pd
data = pd.read_csv('rainfall.csv')

data_cleaned = data.drop(['time'], axis=1)
data_cleaned.groupby(['year', 'month', 'date']).mean()

                 rainfall(cm)
year month date              
2012 12    5             13.0
2013 4     3             16.5
 
1 members found this post helpful.
Old 01-10-2018, 01:36 AM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,126

Rep: Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120
That's pretty impressive ...

ISTR Sefyir pushing this barrow before.
I keep trying to convince myself there must be a reason to learn python ... maybe there is.
 
1 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
In Shell script, csv file is not generating rks10a Programming 10 10-04-2012 12:30 AM
Need shell script help with a CSV file. lothario Linux - Software 2 08-31-2012 11:40 AM
Reading a .CSV file and then calculating average per minute basis in shell script. krishdeeps Linux - Newbie 1 04-23-2010 04:38 PM
Shell Script for CSV file comparision aravind_balan Programming 1 02-17-2009 03:33 AM
Calculate average from csv file in shell script khairilthegreat Linux - Newbie 5 11-21-2007 12:57 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration