Relaying is when someone uses your SMTP server to send mail somewhere else. If whoever sent the emails didn't send them through your server then stopping relaying won't fix the problem. If they are sending them through your server then stopping (or restricting) relaying will help. Can you provide more info on your mail server please?
By the way, you might also look at http://www.openspf.org/
. SPF is a way of preventing return-path address forgery.