ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
awk '/^[^ ]*:$/ { user = ...; next }
'/^ .*:$/ { server = ...; next }
{ workspace = $1;
sum_s[server] += $3;
sum_u[user] += $3;
sum_w[workspace] += $3;
sum_w_s[workspace][server] += $3;
# whatever kind of sum do you want
}
END { loop on servers, users, workspaces and print what you want }
I don't understand anything in your script. This may be my limitation, not yours. This is the entire script I wrote (and ran) to respond to your Original Post.
Code:
#!/bin/bash
# Daniel B. Martin Sep14
#
# To execute this program, launch a terminal session and enter:
# bash /home/daniel/Desktop/LQfiles/dbm1250.bin
#
# This program inspired by:
# http://www.linuxquestions.org/questions/programming-9/
# parse-file-with-awk-and-sum-totals-4175520091/
# File identification
Path=${0%%.*}
InFile=$Path"inp.txt"
OutFile=$Path"out.txt"
echo "Method of LQ Member danielbmartin"
awk '{if (!match(" \t",substr($0,1,1))) {server=$0; next}
if (substr($0,1,1)==" ") {workspace=$0; next}
print server,workspace,$0}' \
$InFile >$OutFile
echo "OutFile ..."; cat $OutFile; echo "End Of File"
echo; echo "Normal end of job.";echo; exit
Daniel B. Martin
Last edited by danielbmartin; 09-26-2014 at 10:51 AM.
Reason: Cosmetic improvement, no change to code
I don't understand anything in your script. This may be my limitation, not yours. This is the entire script I wrote (and ran) to respond to your Original Post.
Code:
#!/bin/bash
# Daniel B. Martin Sep14
#
# To execute this program, launch a terminal session and enter:
# bash /home/daniel/Desktop/LQfiles/dbm1250.bin
#
# This program inspired by:
# http://www.linuxquestions.org/questions/programming-9/
# parse-file-with-awk-and-sum-totals-4175520091/
# File identification
Path=${0%%.*}
InFile=$Path"inp.txt"
OutFile=$Path"out.txt"
echo "Method of LQ Member danielbmartin"
awk '{if (!match(" \t",substr($0,1,1))) {server=$0; next}
if (substr($0,1,1)==" ") {workspace=$0; next}
print server,workspace,$0}' \
$InFile >$OutFile
echo "OutFile ..."; cat $OutFile; echo "End Of File"
echo; echo "Normal end of job.";echo; exit
Daniel B. Martin
Hi Daniel thanks for checking back in. I ran your script against the truncated version of the whole file and I got the same results that you did. I uploaded the input file so you could try it out. It's way too long to paste into a code bracketed field on here.
I modified the script slightly to remove unneeded lines and the ever popular '(LOCKED):' string.
awk -F"[ :]+" 'NF == 2{ user = $1 }
NF == 3{ server = $2 }
NF > 3{ ws = $2;
sum_s[server] += $3;
sum_u[user] += $3;
sum_w[ws] += $3
}
END { loop on servers, users, workspaces and print what you want }' file
The print what you want part is being able to print the top 5 space consumers for each workspace. I can probably figure out how to do that but can you at least check out the version of the input file jm_out that I attached and see what you can do with it? Thanks so much for your interest and help.
awk '/^[^ ]*:$/ { user = ...; next }
'/^ .*:$/ { server = ...; next }
{ workspace = $1;
sum_s[server] += $3;
sum_u[user] += $3;
sum_w[workspace] += $3;
sum_w_s[workspace][server] += $3;
# whatever kind of sum do you want
}
END { loop on servers, users, workspaces and print what you want }
I will try it when I get home tonight I have to invent the 'do what you want' part. Thanks so much for yout help. I've attached the input file 'jm_out'.
Well I am big believer in helping and not doing. So here is a nudge in the right direction. You will need to figure out the logic on how to best sort the data and only show 5 in each section.
Code:
#!/usr/bin/awk -f
BEGIN{ FS = "[ :]+" }
NF == 2{ user = $1 }
NF == 3{ server = $2 }
NF > 3{
ws = $2
sum[server][ws][user] += $3
}
END{
for(i in sum)
{
print "Server: ",i
for(j in sum[i])
{
print "Top 5 space consumers on",j":"
for(k in sum[i][j])
print k," with size ",sum[i][j][k]
}
}
}
I attached the full file finally. I realized that I didn't have the ".txt" extension on the file previously. If you have time and you wouldn't mind, can you try your script on the attached file as the input file? Thanks so much.
So this file is generated with a perl script. I would suggest you to sum up those things in that perl script. In perl you can print the required result easily there is a built-in sort function. (so print what you want means reverse sort list and print the ordered result)
The sample input file contains tab characters. The large "real world" input file does not. Please investigate and reconcile this discrepancy.
Daniel B. Martin
Daniel I know what caused the no tabs. It's the sed in the first line that gets rid of the locked string. I'll get rid of it when I get home. Thanks for pointing that out.
Last edited by master-of-puppets; 09-27-2014 at 01:44 PM.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
