What is the most likely reason why a perl script running on Linux partly fails on FreeBSD?

I have been struggling with a perl script application (Anomy Sanitizer, a great e-mail filter with built-in macro scanner and a possibility of calling external virus scanner) for days now. Previously, I configured it on Linux in an hour, and works well there.
I did everything on FreeBSD as on Linux, yet it does not work. The perl version seems to be OK, all missing perl modules are downloaded, paths and file permissions seem to be OK, yet it does not work: the macro scanning module.and the external virus scanner always return invalid exit codes, therefore all scanned attachments are regarded as infected and go to the quarantine.
Unfortunately, I am not familiar with the perl source code of the program, so I cannot easily debug it and the built-in debugging facilities do not help me much. (Even the supplied debugging shells
produced some extra error messages due to invalid shell syntax "let FAILED=$FAILED+1')
I begin to become very frustrated, please help!

As I stated in your previous post about "shebang"
In addition at the top of every Perl sciprt put:

#!/path/to/perl -w
use strict;
use diagnostics;

This will give you a detail errror output.

Login with your user account
type perl

Adjust "shebang" to output of above command?

chmod +x scriptname

$ ./scriptname

Also:
Post guestion about perl scripting to http://www.perlmonks.org

What does "not running" mean. It error outor just flat out
doesn't do anything?

In the subject I should rather write "... not running well...".
The program receives the e-mails from procmail, re-constructs them and separates the attachment, but the built-in macro scanning module (located in a separate .pl file) and the separate virus scanner always return invalid exit codes. However, all messages are passed through, possibly thanks to procmail.
The application has some debug shells, "test cases" with different reference e-mail messages used as input. Not all of them fails. Those which fail produce .diff files (this contains the differences between the expected and the actual output of the program in response to the reference input e-mail message), and also a .log file, but this is somehow always empty.
Thanks for the "#!/path/to/perl -w" tip, I will try it tommorrow.

Your more than welcome.
I would one again suggest that in addition to post specfic
question about Perl here, that you also post to http://perlmonks.org

And you could always try writing the the author?

I am sure I will visit the perl forum one day, but this time I am not convinced if my problem is perl specific or FreeBSD specific.
I rather suspect it is the latter, since the perl environments are unquestionably the same on FreeBSD and Linux.
I do not go into the very details since there is too much information to be posted here and I do not know what is important and what is not.
Because of this, I do not expect very specific replies, either. I only hope that there are some guys who also had troubles when took something from Linux to FreeBSD and now they know what makes the difference.
The author's mailing list seems to be dead since there was only one posting in the past one year, and even that one was a spam.

I was so curious about this whole thing I downloaded anomy and took a look at it myself. Here is what I changed to make testall.sh pass the tests:

In testall.sh
line 1: Change /bin/sh to /usr/local/bin/bash

In sanitizer.base64.t
line 18: Change /bin/false to /usr/bin/false

In sanitizer.filenames.t
line 18: Change /bin/false to /usr/bin/false

Now after you run ./testall.sh, it will still complain about sanitizer.base64 and sanitizer.filenames failing. BUT if you look at the diffs:

For sanitizer.base64.diff:
Just differences in whitespace, and the difference in /bin/false vs. /usr/bin/false

For sanitizer.defaults.diff:
Just differences in the expected return of trying to remove a file thats not there

So really they aren't failing, its just a slight discrepancy from the output it was expecting.

I didn't actually try to hook anomy into my sendmail or anything, but it does pass the self-test.

Good luck.

Hi, Elm,

I did what you suggested, but it still gives me actual differences in sanitizer.filenames.diff:

144c144
< Unknown exit code: 65280
---
> File was infected, but the virus checker fixed it.
147a148,150
> Match (rule="10"):
> Enforced policy: unknown
>

and the latter message is repeated for each attachment found in the reference file.
Then I turned on diagnostics as Picledbeans suggested and was astonished by the number of "Use of uninitialized value ..." errors found in the .log files. These errors were also found for testcases which did not produce deviating output.
For me, the sanitizer.filenames testcase has the greatest importance, since this gives deviating output.
Here is an extract from the contents of sanitizer.filenames.log:

Use of uninitialized value in pattern match (m//) at ../bin/Anomy/MIMEStream.pm line 1674 (#1)
Use of unitialized value in pattern match (m//) at ../bin/Anomy/MIMEStream.pm line 1675 (#1)
Use of unitialized value in pattern match (m//) at ../bin/Anomy/Sanitizer.pm line 1486 (#1)
Use of unitialized value in concatenation (.) or string at ../bin/Anomy/Sanitizer.pm line 1208 (#1)

and so force.

I will try to debug the code, but it will be hard since I have never seen a perl script before (nor similar). Thanks for your help!

Did you start with a fresh extraction of the anomy tarball and make just those 3 changes?

(btw, the occurance of /bin/false is on line 36 of file sanitizer.base64.t, not line 18 like I mistakenly stated before, but you probably figured that out)

When I make just those 3 changes on a fresh extraction of the tarball and run ./testall.sh, everything passes (including sanitizer.filenames.t) except for the whitespace etc issues with sanitizer.base64 and sanitizer.defaults

What version of anomy are you working with? The one I downloaded is anomy-sanitizer-1.49

How did you install the perl modules you needed (Digest::MD5,MIME::Base64,MIME::QuotedPrint)? By hand or with the ports tree (I used the ports tree). I didn't need to add IO::File, as it was already in the base system.

> Did you start with a fresh extraction of the anomy tarball and make just those 3 changes?

I tried fresh extraction and also copying of the Linux files with the same results.

> (btw, the occurance of /bin/false is on line 36 of file sanitizer.base64.t, not line 18 like I mistakenly stated before, but you probably figured that out)

Yes, I found /bin/false and corrected it.

> When I make just those 3 changes on a fresh extraction of the tarball and run ./testall.sh, everything passes (including sanitizer.filenames.t) except for the whitespace etc issues with sanitizer.base64 and sanitizer.defaults

Not for me on FreeBSD. (But it passed everything on Linux)

> What version of anomy are you working with? The one I downloaded is anomy-sanitizer-1.49

I also use the 1.49 version.

>How did you install the perl modules you needed (Digest::MD5,MIME::Base64,MIME::QuotedPrint)? By hand or with the ports tree (I used the ports tree). I didn't need to add IO::File, as it was already in the base system.

On Linux I used CPAN, but it did not work here on FreeBSD (sure I misconfigured something with CPAN or ftp is forbidden here), so I downloaded and manually installed the ports found at www.freebsd.org.
Things did not change, so I began to download modules from here and there (including IO::File), and finally I got a message 'perl is hashed' when I entered 'type perl'. However, things did not become better or worse ever since perl is 'hashed'.
Then I downloaded the complete perl 5.6.1 tarball and installed that. It did not matter: perl remained 'hashed' and the script does not run correctly.
Summarizing: from the time that I downloaded the missing modules nothing really mattered; the original error remained. It did not matter if I installed other versions of the missing modules, if they were ports or compiled; if I installed a new perl package, if perl was hashed or not.
So I have no clue, and, for the time being I am horrified by perl language.
By first glance, it will take days just to find out what is a variable what is object, function, etc. And there is still a long way to go from there.

OK, we have an other FreeBSD 4.4-stable server, I installed the stuff there, and it does not work there, either!

Made CPAN work, let it download the complete perl 5.6.1 release; downloaded the missing modules; doublechecked them; yet it does not work.

Above all, I came to realize that the "Uninitialized value ..." errors do not matter: they are also present in the same number on the Linux installation, where there is no problem.

Shit

The perl I used was that which came with the base system: version 5.005_03. The perl binary is at /usr/bin/perl (and a symlink from /bin/perl to it).

If you install a new perl, it probably was installed at /usr/local/bin/perl.

Similarly, the modules I installed via the ports system (e.g., "cd /usr/ports/converters/p5-MIME-Base64 && make install") were placed in /usr/local/lib/perl5/site_perl/5.005

If you installed a new perl, and then installed modules for it, they probably went somewhere else.

None of that may matter though, if testall.sh isn't complaining about missing modules.

As an aside, use the port system if at all possible to install new stuff on FreeBSD. It can make a difference to a lot of programs, keeps things tidy, and make upgrading those programs later easy. Unfortunately anomy isn't in the ports system, but most perl modules and the latest perl version are.

You put anomy on the 4.4-stable, installed the needed modules, made the 3 changes I suggested, and testall.sh complained of failures and those failures weren't just whitespace differences or something else insignificant? And then you install perl 5.6.1 and modules and got the same result? Or did you install perl 5.6.1 and modules before trying testall.sh the first time on the 4.4-stable server?

The reason I ask is because the system I tried it on is pretty much stock 4.6-release, which has the same perl version and base modules as 4.4-stable. The only modules I added were the ones needed by anomy (by doing a "make install" in /usr/ports/converters/p5-MIME-Base64 and in /usr/ports/security/p5-Digest-SHA1). I made no other changes to the perl environment at all. And testall.sh passed everything (except for the whitespace and other insignificant issues for base64.t and defaults.t). If its acting differently for you, I can only guess that it is because of a change you made to the perl environment or anomy scripts that I did not.

You said that filenames.t was still failing. Are you absolutely sure you changed /bin/false to /usr/bin/false? Are you absolutely sure you didn't accidentally erase one of the surrounding colons or quote-marks? The only way I can make filenames.t fail like you saw is by leaving it as /bin/false or by screwing up the surrounding punctuation.

...can't think of anything else at the moment...

Hi, elb,

Finally I got it working!

It was a silly thing: those .t files in the testcases directory turned out to be shells called within testall.sh.
Each of those .t shells began as follows:
#!/bin/sh
You can find out the end of the story.
It is a shame that it took me three days to figure this out...

Thank you for your attention to the problem.

By the way, you have you tried anomy with sendmail? What is your opinion about it?

No, I didn't hook it into sendmail. I just downloaded it because I was curious about the trouble you were having.

My sendmail installation is already so customized I'm almost afraid to touch it anymore or it might go completely fubar. Heh...