I'm trying to set up a mail server only for a network so that the users of the network can email each other. I know it sounds kind of pointless, but I'm behind a firewall that doesn't allow to forward my mail server and I can't do anything about it.I connected with outlook to my mail server and it doesn't work. It says that "The hostname mycomp.mydomain" cannot be found. Please verify that you've entered the server name correctly". If I enter my IP instead of mycomp.mydomain everything works fine. Could you please help me if you can

go on using ip of server instead of its name. if u have a email server which works for local network, u dont need DNS mx record. so it is not issue.
if u really want local clients to reach email server via its name, u must configure bind (named). if u have already a nameserver, create an A and MX record for email server in the zone file.