Yes and no
Depending on what hardware you have at your house, there are various ways that you could handle this. From what your message said, I gather that your concern in this area centers mainly around transactions that are going to be going over your [voice] line. In order for your modem to be able to handle both data and VOICE transactions, theoretically you need to have a special VOICE modem installed (probably going to cost a little more than your typical Winmodem)
In addition to this, your local telephone company has special rings available (at some additional charge, of course) that can inform you of whether you are receiving 1) a VOICE call 2) a DATA call, or 3) a FAX. (Whatever software you are using for this should have a section where you can set this up if you have purchased this service)
Myself, I have fantasized about setting up something nice like this at my place, but I was a little disappointed to discover that all of the modems that I own (many of them second hand) seem to be only data/fax equipped. Something that I did notice about this, that has to do with the fact that when my data only modem receives a voice call, I can hear what the person on the other end is saying, and my research into Hayes compatible modem commands, I think that it may be theoretically possible to get around this. Especially given the speed and efficiency of most modern computers.
I happen to have at my house a nifty little gadget that goes from telco connector to 1/8¨ mono headphone type audio connector that I got from Radio Shack. I thought about trying to write a script that would record a message onto my soundcard (after I connected my modem to it with this), however after realizing that this only goes one way, I decided not to waste the time. Theoretically, with a couple of these, one going each way, and some scripts to properly synchronize your soundcard and recording software (should your data/fax modem fail to connect within like 2 seconds) you could accomplish something a lot like this without the special modem or specialized ring.
If I were you, and you have serious customers that might be on the line with a setup like this, I would just go ahead and SPEND the $$$ for the true VOICE modem and specialized telco rings. If anyone works out anything like this on their own, please let me know!!!