I checked relay system using IP address it working gr8, but I want to implement user based authorization in relay server. but no idea where to make changes in sendmail.mc.. and my outlook asking for password again and again..

any help guys

whats role of openssl in user based authentication ?

telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 smtest.goforhosting.com.178.118.111.in-addr.arpa ESMTP Sendmail 8.13.8/8.13.8; Sat, 19 Mar 2011 11:13:07 -0400
ehlo localhost
250-smtest.goforhosting.com.178.118.111.in-addr.arpa Hello localhost [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP

It looks like the authentication should work. It would be useful to find out what authentication actually fails with this:
If you note "your outlook ..." it means you are trying to configure SMTP on some client MS PC. But the telnet test above is from localhost. It would be useful to try it from the outlook client PC command-line. (You can get different result)

Please post some logs from maillog about the unsuccessful authentication.
It secures your connection to SMTP before you send user names and passwords.