no software neccessary, just try to send an email from outside the relay, to someone else outside the relay eg: if your relay is mydomain.com, telnet to your mailserver(from an outside connection) and attempt to send a message to
youraddress@another.domain.com, if it gets through then you have an open relay.