LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-26-2011, 11:59 AM   #1
rwallingford
LQ Newbie
 
Registered: Apr 2011
Posts: 2

Rep: Reputation: 0
Using Array variables


I'm a relative new bash scripting guy, here's my dilemma.

I need to parse through a file which contains timestamps of transactions. What I am trying to do is come up with a Max Transactions Per Second (TPS) value. I was thinking that creating an array variable would be the way to go, but I'm having problems determining even how to start.

Any tips/tricks would be great.

Thanks in advance,
Rich
 
Old 04-26-2011, 12:54 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,251

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Maybe if you show us some of the data we can assist? Remember to not only include the input file but also what your desired output would look like based on the data you provide.
 
1 members found this post helpful.
Old 04-26-2011, 01:42 PM   #3
rwallingford
LQ Newbie
 
Registered: Apr 2011
Posts: 2

Original Poster
Rep: Reputation: 0
more detail

I have one app that runs on 6 servers, with 5 instances on each server. I'm interested in parsing one log file which contains timestamp data. An excerpt of that file is included here. These files contain several thousand lines each. I am only interested in the timestamp column, and specifically just the hr/min/sec fields of that.

As far as output, I'd like to see the Max Transactions per Second - so, how many transactions occured and at what time. I'm very open on the look and feel.

.060, 2011/04/26, 10:20:09.223, /rws/services/Monitoring,
.012, 2011/04/26, 10:20:09.673, /rws/services/Monitoring,
.388, 2011/04/26, 10:20:10.753, /rws/services/Cardholder200702,
.012, 2011/04/26, 10:20:11.473, /rws/services/Monitoring,
.061, 2011/04/26, 10:20:11.742, /rws/services/Monitoring,
.218, 2011/04/26, 10:20:12.813, /rws/services/Cardholder2009,
.290, 2011/04/26, 10:20:16.700, /rws/services/Cardholder2009,
.060, 2011/04/26, 10:20:19.223, /rws/services/Monitoring,
.012, 2011/04/26, 10:20:19.703, /rws/services/Monitoring,
.012, 2011/04/26, 10:20:21.503, /rws/services/Monitoring,
.062, 2011/04/26, 10:20:21.702, /rws/services/Monitoring,
.215, 2011/04/26, 10:20:23.347, /rws/services/Cardholder2009,
.267, 2011/04/26, 10:20:23.373, /rws/services/Statements2010,
.061, 2011/04/26, 10:20:29.233, /rws/services/Monitoring,
.012, 2011/04/26, 10:20:29.693, /rws/services/Monitoring,
 
Old 04-26-2011, 02:38 PM   #4
arizonagroovejet
Senior Member
 
Registered: Jun 2005
Location: England
Distribution: openSUSE, Fedora, CentOS
Posts: 1,078

Rep: Reputation: 195Reputation: 195
This is rather quick and dirty, 'using array variables' doesn't come in to it, there might a a useless use of cat award in there, but it is bash scripting


Code:
$ cat file | cut -d , -f 3 | cut -d . -f 1 | sort | uniq -c | sort -n -r | head -1 | sed 's/^ *//' | cut -d ' ' -f 1
If you want to understand what it's doing, then run start off just running the command up to the second | and then add on the other commands one at a time. Look at the man pages for cut sort uniq and head to see what the options do.

With the data sample you provided the number that comes out is 2, which can easily verify is correct by looking at the data since there's not very much of it.

I expect someone can come up with something more elegant.

Arguably this would be better achieved with a scripting language like perl or python.
 
Old 04-26-2011, 09:19 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,251

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Well there would need to be more work done to break down by change in minute and hour, but this may help:
Code:
awk -F"[\t ]*,[\t ]*" '{split($3,f,"[:.]");sec[f[3]]++}END{n = asort(sec);print sec[n]}' file
hours and minutes are f[1] and f[2] respectively.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
PHP Accesing variables within an array LionOfGod Programming 3 03-31-2011 06:43 AM
bash: use file as input into array, parse out other variables from array using awk beeblequix Linux - General 2 11-20-2009 11:07 AM
Adding array variables together nobody123 Linux - Newbie 2 02-26-2009 05:43 PM
array variables from html form not processed hil4d Linux - Server 2 04-20-2008 11:11 PM
BASH - Array Variables Micro420 Programming 5 12-15-2006 06:49 PM


All times are GMT -5. The time now is 09:05 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration