LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 04-12-2012, 07:23 PM   #16
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Distribution: UBUNTU 5.10 since Jul-18,2006 on Intel 820 DC
Posts: 618

Rep: Reputation: 136Reputation: 136

Incidentally, why excel?

Could well have been achieved in a linux environment itself using OO, Libre, gnumeric etc.

OK
 
Old 04-12-2012, 08:43 PM   #17
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,083

Rep: Reputation: 287Reputation: 287Reputation: 287
A round-up of test results.

Here are the five input files. The numeric values were chosen arbitrarily.

Code:
/home/daniel/Desktop/LQfiles/dbm330i01.txt
100 99
300 900
500 2500
200 200
200 8000

/home/daniel/Desktop/LQfiles/dbm330i02.txt
55 133
121 139
117 44
88 117
109 108

/home/daniel/Desktop/LQfiles/dbm330i03.txt
66 133
133 139
118 144
100 167
101 122

/home/daniel/Desktop/LQfiles/dbm330i04.txt
67 141
104 132
112 135
131 148
104 129

/home/daniel/Desktop/LQfiles/dbm330i05.txt
111 128
115 142
127 89
138 141
114 139
Here are the code examples and the results they produced.


Code:
echo
echo "Method of LQ guru grail (using awk)"
[ -e "$OutFile01" ] && rm "$OutFile01"
for file in /home/daniel/Desktop/LQfiles/dbm330i*.txt; do
awk '$2 < 100{tot[FILENAME]+= $2 / $1;count[FILENAME]++}END{for(f in tot)print f,tot[f]/count[f]}' $file >> $OutFile01
done

/home/daniel/Desktop/LQfiles/dbm330i01.txt 0.99
/home/daniel/Desktop/LQfiles/dbm330i02.txt 0.376068
/home/daniel/Desktop/LQfiles/dbm330i05.txt 0.700787
Code:
echo
echo "Method of LQ member millgates (using awk),"
echo " with suggested improvements of LQ guru grail"
[ -e "$OutFile02" ] && rm "$OutFile02"
for file in /home/daniel/Desktop/LQfiles/dbm330i*.txt; do
  awk -v ifname="$file" -v ofname="$OutFile02" '
    BEGIN { sum = 0; num = 0; OFS="\t" }
    $2 > 100 { x=$2/$1; sum+=x; num++;  print $1"\t"$2"\t", x | "sort -nk 2" }
    END { print ifname, sum/num >> ofname }
    ' "$file" > "${ifname}.sorted"
done

/home/daniel/Desktop/LQfiles/dbm330i01.txt      12.25
/home/daniel/Desktop/LQfiles/dbm330i02.txt      1.47183
/home/daniel/Desktop/LQfiles/dbm330i03.txt      1.4317
/home/daniel/Desktop/LQfiles/dbm330i04.txt      1.38984
/home/daniel/Desktop/LQfiles/dbm330i05.txt      1.15724
Code:
echo
echo "Method of LQ guru grail (using bash)"
[ -e "$OutFile03" ] && rm "$OutFile03"
for f in /home/daniel/Desktop/LQfiles/dbm330i*.txt
do
    tot=0
    count=0
    while read -r x y
    do
	if (( y < 100 ))
	then
	    tot=$( echo "$tot + $y / $x" | bc -l )
	    (( count++ ))
	fi
    done<"$f"
    mean=$( echo "$tot / $count" | bc -l )
    echo -e "$f\t$mean" >> $OutFile03
done

/home/daniel/Desktop/LQfiles/dbm330i01.txt      .99000000000000000000
/home/daniel/Desktop/LQfiles/dbm330i02.txt      .37606837606837606837
/home/daniel/Desktop/LQfiles/dbm330i03.txt      
/home/daniel/Desktop/LQfiles/dbm330i04.txt      
/home/daniel/Desktop/LQfiles/dbm330i05.txt      .70078740157480314960
Code:
echo
echo "Method of LQ member millgates (using sed+bc)"
files=( /home/daniel/Desktop/LQfiles/dbm330i*.txt )
[ -e "$OutFile04" ] && rm "$OutFile04"
for f in /home/daniel/Desktop/LQfiles/dbm330i*.txt; do
    echo -e "$f\t$((sed -r 's_([0-9]+)\s([0-9]+)_if(\2>100){sum+=\2/\1;cnt+=1}_' "$f";echo "sum/cnt")|bc -l)" >> $OutFile04
done

/home/daniel/Desktop/LQfiles/dbm330i01.txt      12.25000000000000000000
/home/daniel/Desktop/LQfiles/dbm330i02.txt      1.47182832284479490484
/home/daniel/Desktop/LQfiles/dbm330i03.txt      1.43170481444729154959
/home/daniel/Desktop/LQfiles/dbm330i04.txt      1.38984422635584763874
/home/daniel/Desktop/LQfiles/dbm330i05.txt      1.15724328447440575586
The results are not all alike. For those who contributed code: please examine my rendition of your post to make sure I didn't botch it.

Daniel B. Martin
 
Old 04-12-2012, 10:04 PM   #18
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by danielbmartin View Post
The results are not all alike. For those who contributed code: please examine my rendition of your post to make sure I didn't botch it.
You really botched mine! Here's its output with your example data (after adapting it to use space instead of comma to parse input):
Code:
"dbm330i01.txt",12.25
"dbm330i02.txt",1.47182832284479
"dbm330i03.txt",1.43170481444729
"dbm330i04.txt",1.38984422635585
"dbm330i05.txt",1.15724328447441
Does that mean I win?
Kevin Barry
 
Old 04-13-2012, 12:30 AM   #19
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,471

Rep: Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888
Well for mine you can see the 2 scripts output the same except digits after the decimal point, an easy fix on either side. The reason they will differ to the others is I used y < 100
and they used y > 100. This was driven from the second requirement:
Quote:
2) Delete all data points whose y-coordinate (column 2) was less than a specified value, in this case 100.
I of course read this wrong (or too quickly as is normally the case) and saw 100 and less than

So to have mine concur with the others is again a simple change:
Code:
#Awk
awk '$2 >= 100{tot[FILENAME]+= $2 / $1;count[FILENAME]++}END{for(f in tot)print f,tot[f]/count[f]}' /home/daniel/Desktop/LQfiles/dbm330i*.txt

#Bash
#!/bin/bash

for f in /home/daniel/Desktop/LQfiles/dbm330i*.txt
do
    tot=0
    count=0
    while read -r x y
    do
	if (( y >= 100 ))
	then
	    tot=$( echo "scale=6; $tot + $y / $x" | bc )
	    (( count++ ))
	fi
    done<"$f"
    mean=$( echo "scale=6; $tot / $count" | bc )
    echo -e "$f\t$mean" >> $OutFile03
done
 
1 members found this post helpful.
Old 04-13-2012, 09:20 AM   #20
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,083

Rep: Reputation: 287Reputation: 287Reputation: 287
Test results, redux.

Thank you, grail, for minor corrections to your code. With those changes all results are equivalent.

Apologies to you, ta0kira, for omitting your code. No offense intended. As an inexperienced player in the Linux world, I had never even heard of Rscript. Consequently I was unable to understand your code, unable to execute it.

Let's hope that OP benefits from the many ideas presented in this thread. For sure, I did.

Thanks to all who contributed in any fashion.

Daniel B. Martin
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Pipelining data of find command in an array after processing the contents of the data AshishJogeshwar Linux - Software 5 06-10-2010 01:52 AM
Pipelining data of find command in an array after processing the contents of the data AshishJogeshwar Linux - Software 0 06-07-2010 06:15 AM
automate SSL certificate generation jakev383 Linux - Server 4 07-24-2009 08:50 AM
Processing data from a 'foreign' database with mysql, or tools to pre-process data. linker3000 Linux - Software 1 08-14-2007 08:36 PM
perl CGI + dynamic table generation kurrupt Programming 8 09-27-2006 10:25 PM


All times are GMT -5. The time now is 02:02 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration