Greetings,
Well, I would start with the logs and see what they can tell you. If they don't seem to tell you much, increase verbosity to 'info' level if you can.
Now, as far as ideas for configuring sendmail on (Open)Solaris, have you seen
James Liu's blog on Sendmail configuration in Solaris? He breaks down the configuration for a low-volume sendmail server config pretty well.
Admittedly, I haven't had much contact with
Solaris/
OpenSolaris in the last several years, but I did drop by the
Illumos breakout session at
SCaLE last weekend and was pleased to see that the project is still alive and well.
So, I hope that helps. Let us know.