perl newbie

mattca · 10-22-2010, 09:27 PM

Hi all, I wrote a simple script to read in a list of transactions from my online banking in CSV format, scan it for keywords associated with budgets, and then provide the totals.

It works, but I'm wondering if there is a more natural way to do any of this with perl. Any suggestions appreciated!

Code:

#!/usr/bin/perl

use strict;
use warnings;

my %budgets;
my $budget;
my $transactions_file = $ARGV[0] or die "usage: calc.pl <transactions file>\n";
my %transactions;
my %totals;

my $first_transaction;
my $last_transaction;

$budgets{Groceries} = ["NO FRILL'S", "SOBEYS", "RABBA", "VALUMART", "LONGO"];
$budgets{Gas} = ["ESSO", "PETRO"];
$budgets{Savings} = ["<snip>"];
$budgets{"Eating out"} = ["subway", "wrap and grab", "bbq", "pizza", "cafe"];
$budgets{Alcohol} = ["lcbo", "beer"];
$budgets{"Drug store"} = ["shoppers", "rexal"];
$budgets{Office} = ["staples"];
$budgets{Walmart} = ["wal-mart"];
$budgets{Telus} = ["telus"];
$budgets{Rogers} = ["rogers"];
$budgets{Hydro} = ["tor hyd elec"];
$budgets{"Bank fees"} = ["fee", "interest"];
$budgets{"Visa - TD"} = ["tfr-to visa"];
$budgets{"Visa - Desjardins"} = ["visa desjard"];
$budgets{Pets} = ["pet"];
$budgets{Cash} = ["w/d"];
$budgets{Cheques} = ["chq"];
$budgets{"Canadian Tire"} = ["canadian tire"];
$budgets{Motorcycle} = ["cycle"];

open TRANSACTIONS, "$transactions_file" or die "can't open $transactions_file ($!)\n";
my @all_transactions = <TRANSACTIONS>;
close (TRANSACTIONS);

if ($all_transactions[0] =~ /^([^,]*),/) {
	$first_transaction = $1;
}

if ($all_transactions[@all_transactions - 1] =~ /^([^,]*),/) {
	$last_transaction = $1;
}

foreach $budget (keys %budgets) {
	my @t;

	foreach my $keyword (@{$budgets{$budget}}) {
		@t = (@t, grep /$keyword/i, @all_transactions);
	}

	$transactions{$budget} = \@t;

	$totals{$budget} = 0;

	foreach (@{$transactions{$budget}}) {
		my @bits = split /,/;

		if ($bits[2]) {
			$totals{$budget} += $bits[2];
		}

		if ($bits[3]) {
			$totals{$budget} -= $bits[3];
		}
	}

	$totals{$budget} = '$' . sprintf "%.2f", $totals{$budget};
}

$" = "";

print "First transaction: $first_transaction, Last transaction: $last_transaction\n\n";

print "Raw transactions:";

foreach $budget (sort keys %transactions) {
	$" = "\t";
	print "\n$budget:\n\t@{$transactions{$budget}}";
}

print "\nTotals:\n";
foreach $budget (sort keys %totals) {
	print "\t$budget: $totals{$budget}\n";
}

Sergei Steshenko · 10-22-2010, 10:01 PM

In your case you can write:

Code:

my %budgets =
  (
  Groceries => ["NO FRILL'S", "SOBEYS", "RABBA", "VALUMART", "LONGO"],
  Gas => ["ESSO", "PETRO"],
  Savings => ["<snip>"],
  'Eating out' = ["subway", "wrap and grab", "bbq", "pizza", "cafe"],
  ...
  );

...

Regarding

Code:

open TRANSACTIONS

- as a habit don't use global filehandles; "$transactions_file" is no better than $transactions_file, better write

Code:

open(my $transcations_fh, '<', $transactions_file) or die "can't open '$transactions_file' file for reading ($!)\n";

.

mattca · 10-22-2010, 10:14 PM

Thanks Sergei!

Quote:

Originally Posted by Sergei Steshenko

In your case you can write:

Code:

my %budgets =
  (
  Groceries => ["NO FRILL'S", "SOBEYS", "RABBA", "VALUMART", "LONGO"],
  Gas => ["ESSO", "PETRO"],
  Savings => ["<snip>"],
  'Eating out' = ["subway", "wrap and grab", "bbq", "pizza", "cafe"],
  ...
  );

I was aware that I could have written it that way, but find the other way a bit more readable.

Quote:

Code:

open TRANSACTIONS

- as a habit don't use global filehandles; "$transactions_file" is no better than $transactions_file, better write

Code:

open(my $transcations_fh, '<', $transactions_file) or die "can't open '$transactions_file' file for reading ($!)\n";

.

Oh, thank you. I found the code for reading in a file through google, and didn't know that I was using a global filehandle.

So using your example, I can do something like this instead:

Code:

open(my $transcations_fh, '<', $transactions_file) or die "can't open '$transactions_file' file for reading ($!)\n";
my @all_transactions = <$transactions_fh>;

Is that correct?

Thanks!

Sergei Steshenko · 10-22-2010, 10:27 PM

Quote:

Originally Posted by mattca

Thanks Sergei!

I was aware that I could have written it that way, but find the other way a bit more readable.

Oh, thank you. I found the code for reading in a file through google, and didn't know that I was using a global filehandle.

So using your example, I can do something like this instead:

Code:

open(my $transcations_fh, '<', $transactions_file) or die "can't open '$transactions_file' file for reading ($!)\n";
my @all_transactions = <$transactions_fh>;

Is that correct?

Thanks!

Readability is always debatable; my code has less '=' which, I think, makes it better.

The idiom you've used sometimes must be used - when there is inter-key dependency, e.g.

Code:

my %hash;

$hash{number} = 3;
$hash{increased_by_2_number} = $hash{number} + 2;

.

Yes, you are dealing with the newly introduced $transcations_fh correctly.

You can have your whole hash in a separate file:

Code:

sergei@amdam2:~/junk> cat -n data_importer.pl
     1  #!/usr/bin/perl -w
     2
     3  use strict;
     4
     5  my $hash_ref = require "./data.prl";
     6
     7  foreach my $key(sort keys %{$hash_ref})
     8    {
     9    my $value = ${$hash_ref}{$key};
    10    warn "\$key=$key, |\$value|=|$value|"; # '|'s show the limits for clarity
    11    }
    12
sergei@amdam2:~/junk> cat -n data.prl
     1  use strict;
     2
     3
     4  # anonymous hash reference to be exported
     5  {
     6  foo => "whatever you like",
     7  bar => "and something you need"
     8  };
sergei@amdam2:~/junk> ./data_importer.pl
$key=bar, |$value|=|and something you need| at ./data_importer.pl line 10.
$key=foo, |$value|=|whatever you like| at ./data_importer.pl line 10.
sergei@amdam2:~/junk>

.

ghostdog74 · 10-22-2010, 10:47 PM

@OP, can you show a sample of your transaction file?

mattca · 10-22-2010, 11:22 PM

Quote:

Originally Posted by Sergei Steshenko

You can have your whole hash in a separate file:

Thanks, yeah I was planning on moving it to a separate file when I cleaned it up, thanks for showing me how to do that!

mattca · 10-22-2010, 11:26 PM

Quote:

Originally Posted by ghostdog74

@OP, can you show a sample of your transaction file?

sample debit transaction:

Code:

09/23/2010,NO FRILL'S #130,34.49,,608.83

sample credit transaction:

Code:

09/22/2010,HQ003 TFR-FR <snip>,,700.00,699.20

The basic format is:

Code:

<date>,<description>,<debit>,<credit>,<balance>

Sergei Steshenko · 10-22-2010, 11:30 PM

Quote:

Originally Posted by mattca

sample debit transaction:

Code:

09/23/2010,NO FRILL'S #130,34.49,,608.83

sample credit transaction:

Code:

09/22/2010,HQ003 TFR-FR <snip>,,700.00,699.20

The basic format is:

Code:

<date>,<description>,<debit>,<credit>,<balance>

Then what does the item in red mean ?

Sergei Steshenko · 10-22-2010, 11:32 PM

Nothing - found an answer.

mattca · 10-22-2010, 11:32 PM

Quote:

Originally Posted by Sergei Steshenko

Then what does the item in red mean ?

It means the purchase was made at No Frills store #130.

mattca · 10-22-2010, 11:34 PM

Quote:

Originally Posted by Sergei Steshenko

And what is your script supposed to do in general ?

Total the debits and credits for a particular budget, based on the assigned keywords.

May do more with it later, but this is it for now.

Sergei Steshenko · 10-22-2010, 11:43 PM

Rereading the initial post and looking at the code I'm thinking you should organize your 'budgets' differently.

If I understand correctly looking at, say,

Code:

$budgets{Groceries} = ["NO FRILL'S", "SOBEYS", "RABBA", "VALUMART", "LONGO"];

'Groceries' is category and

Code:

"NO FRILL'S", "SOBEYS", "RABBA", "VALUMART", "LONGO"

is a list of merchants spendings at which you consider to be spendings on 'Groceries'.

So, I would rather organize the configuration data as

Code:

my %merchants_to_categories =
  (
  "NO FRILL'S" => 'Groceries',
  SOBEYS => 'Groceries',
  ...
  ESSO => 'Gas',
  PETRO => 'Gas',
  ...
  );

- I think the overall code will be simpler then.

Sergei Steshenko · 10-22-2010, 11:45 PM

Quote:

Originally Posted by mattca

It means the purchase was made at No Frills store #130.

So, is the store number to be ignored ? Does the store number always have such format ? Are there any other than numbers store "tags" ?

mattca · 10-22-2010, 11:48 PM

Also, is there a way of sorting a hash by keys?

ie, rather than sorting by keys when outputting:

Code:

#create budgets hash
foreach $budget (sort keys %budgets) {
	# output
}

can I do something like this:

Code:

#create budgets hash
%budgets = <sort %budgets by keys>;

foreach $budget (keys %budgets) {
	# output
}

It would be nice if I could just reorder the hash after I read it in, and not have to use "sort keys" each time it's used later on.

Sergei Steshenko · 10-22-2010, 11:55 PM

Quote:

Originally Posted by mattca

Also, is there a way of sorting a hash by keys?

ie, rather than sorting by keys when outputting:

Code:

#create budgets hash
foreach $budget (sort keys %budgets) {
	# output
}

can I do something like this:

Code:

#create budgets hash
%budgets = <sort %budgets by keys>;

foreach $budget (keys %budgets) {
	# output
}

It would be nice if I could just reorder the hash after I read it in, and not have to use "sort keys" each time it's used later on.

???

Code:

my @sorted_keys = sort keys %hash; # do once

foreach my $key(@sorted_keys) # do as many times as needed
  {
  ...
  $foo = $hash{$key};
  ...
  }

.