perl newbie - can this script be improved?

mattca · 10-22-2010, 11:55 PM

Quote:

Originally Posted by Sergei Steshenko

'Groceries' is category and

Code:

"NO FRILL'S", "SOBEYS", "RABBA", "VALUMART", "LONGO"

is a list of merchants spendings at which you consider to be spendings on 'Groceries'.

In this case, yes, but not in general. For example, transfers to savings are matched with the account number. It is a list of keywords - ie, search terms.

Quote:

So, I would rather organize the configuration data as

Code:

my %merchants_to_categories =
  (
  "NO FRILL'S" => 'Groceries',
  SOBEYS => 'Groceries',
  ...
  ESSO => 'Gas',
  PETRO => 'Gas',
  ...
  );

- I think the overall code will be simpler then.

Hmmm, I'll have to take a look at that tomorrow after I've had some sleep.. but at first glance, I think that would complicate things. For one thing, I'd have a lot more information to keep track of in the "budgets" hash. "Groceries" would be repeated for each and every associated keyword. I think <budget> => <list of keywords> more naturally represents what the data actually is, but I'll try it the other way tomorrow and see if that's simpler.

Thanks again!

Sergei Steshenko · 10-23-2010, 12:00 AM

Quote:

Originally Posted by mattca

In this case, yes, but not in general. For example, transfers to savings are matched with the account number. It is a list of keywords - ie, search terms.

Hmmm, I'll have to take a look at that tomorrow after I've had some sleep.. but at first glance, I think that would complicate things. For one thing, I'd have a lot more information to keep track of in the "budgets" hash. "Groceries" would be repeated for each and every associated keyword. I think <budget> => <list of keywords> more naturally represents what the data actually is, but I'll try it the other way tomorrow and see if that's simpler.

Thanks again!

If you think about it, suggested by me %merchants_to_categories can be obtained from what you have by a dedicated piece of code pretty easily.

My point in general is that there formats convenient for humans (minimum writing required) and there are formats convenient for the CPU (minimum calculations required), so for efficient problem solving give input in the former formats and then during calculations use the latter.

mattca · 10-23-2010, 12:00 AM

Quote:

Originally Posted by Sergei Steshenko

???

Code:

my @sorted_keys = sort keys %hash; # do once

foreach my $key(@sorted_keys) # do as many times as needed
  {
  ...
  $foo = $hash{$key};
  ...
  }

.

Thanks, that helps. Not exactly what I was looking for though. I want to *replace* %hash with a new sorted hash;

ie, something like this:

Code:

%hash = sortHashByKeys(%hash);

foreach my $key (keys %hash) {
    # already in sorted order
}

Your suggestion introduces a new variable to keep track of. I want to use the same variable, but put the keys in sorted order. Is this possible?

Thanks!

mattca · 10-23-2010, 12:07 AM

Quote:

Originally Posted by Sergei Steshenko

My point in general is that there formats convenient for humans (minimum writing required) and there are formats convenient for the CPU (minimum calculations required), so for efficient problem solving give input in the former formats and then during calculations use the latter.

At this stage of the game (my first complete perl script) efficiency is not even on my radar, nor will it matter in this case. I'm much more concerned about having to update "Groceries" in multiple places, should I want to change it to "Food", than I am about how long the calculations will take. With the amount of data I'm dealing with here, any differences in efficiency will not be noticeable anyway.

But thanks for the suggestion, I'll take a look at what the code would look like later and keep efficiency in mind for the future.

Sergei Steshenko · 10-23-2010, 12:15 AM

Quote:

Originally Posted by mattca

Thanks, that helps. Not exactly what I was looking for though. I want to *replace* %hash with a new sorted hash;

ie, something like this:

Code:

%hash = sortHashByKeys(%hash);

foreach my $key (keys %hash) {
    # already in sorted order
}

Your suggestion introduces a new variable to keep track of. I want to use the same variable, but put the keys in sorted order. Is this possible?

Thanks!

IIRC, there is a Perl module implementing hashes with fixed order keys.

But if you insist on fixed order of keys, this means you also have fixed order of values, and altogether after fixing the order you do not need the hash in the first place, i.e. you can simply use:

Code:

my @key_value_pairs_array =
  (
  $key1,
  $value1,
  $key2,
  $value2,
  ...
  $keyN,
  $valueN
  );

.

Sergei Steshenko · 10-23-2010, 12:19 AM

By the way, speaking about efficiency I not only meant CPU usage, I also meant (possibly repeated) data transformations which make code understanding harder.

Sergei Steshenko · 10-23-2010, 12:34 AM

Do you have error checking ? I mean, what do you do if a merchant in your transactions file is unknown to your script ?

Sergei Steshenko · 10-23-2010, 02:13 AM

Thinking about sorting hash keys in your program - I think you need to do it only once - when you output the results in human-readable form. I.e. I don't think you need to sort hash keys for calculations proper.

mattca · 10-23-2010, 11:59 AM

Quote:

Originally Posted by Sergei Steshenko

IIRC, there is a Perl module implementing hashes with fixed order keys.

But if you insist on fixed order of keys, this means you also have fixed order of values, and altogether after fixing the order you do not need the hash in the first place, i.e. you can simply use:

I don't mean fixed order keys.. I am just looking for a way to sort a hash by keys. ie, hashes can have keys in arbitrary order, and I can choose to sort one of them so that each time I want to access it later in the script it's already in sorted order, without using sort keys each and every time.

For example, in PHP I'd do this:

Code:

<?php
$fruits = array("d" => "lemon", "a" => "orange", "b" => "banana", "c" => "apple");
ksort($fruits);
foreach ($fruits as $key => $val) {
    echo "$key = $val\n";
}
?>

Which would output:

Code:

a = orange
b = banana
c = apple
d = lemon

Does perl have a method of doing this - yes or no. I am interested in the answer to my question, not in the different ways of avoiding the action.

Quote:

By the way, speaking about efficiency I not only meant CPU usage, I also meant (possibly repeated) data transformations which make code understanding harder.

I see what you mean now, with the data set up the way you describe I'd only need one loop to perform the calculations. But, I'd need to either a) set up the data that way and repeat the same labels many times, or b) use a nested loop to create the structure you describe. I don't see how that helps.

Quote:

Do you have error checking ? I mean, what do you do if a merchant in your transactions file is unknown to your script ?
....
Thinking about sorting hash keys in your program - I think you need to do it only once - when you output the results in human-readable form. I.e. I don't think you need to sort hash keys for calculations proper.

It seems like we're getting away from perl-specific suggestions and moving toward general programming suggestions. I know how to code, I don't need help with that. I need to know if there are more natural ways of doing what I've done with perl specifically. I can write code, but I am not necessarily aware of the best way to do things with perl.

Do you have any perl-specific suggestions?

Thanks.

Sergei Steshenko · 10-23-2010, 04:27 PM

Quote:

Originally Posted by mattca

...
Do you have any perl-specific suggestions?
...

Search CPAN.

Sergei Steshenko · 10-23-2010, 05:05 PM

The whole

Code:

foreach $budget (sort keys %transactions) {
	$" = "\t";
	print "\n$budget:\n\t@{$transactions{$budget}}";
}

print "\nTotals:\n";
foreach $budget (sort keys %totals) {
	print "\t$budget: $totals{$budget}\n";
}

piece can be replaced by a call to

Code:

Data::Dumper

.