What I look at when choosing or recommending an open source business solution.
What is the overall health of the project.
Is it active, how many minor and major updates have been released?
How many developers are contributing, is the project going to die if the lead developer stops?
Does it have the needed features
Is there a road map published? This will help me understand the direction the project is going in.
Does it have a stable release with the needed features?
Are there any references
Is anybody in the same line of business using the solution? A quick call can tell you of any problems, any adaptions needed etc.
If it is a web application, is there a stable demo site? Look around, see how you could adapt the solution to your needs.
Could you imagine it being used at your company?
Is the solution dependent on other solutions? Are they already on your network or will you have to install them too?
How are the dependencies licensed? There are open source solutions that are dependent on proprietary licensed software.
Documentation and support
What documentation is available?
Is it in tune with your knowledge of computing? Most of us are non coders, some documentation requires programming skills.
Can you get support? What kind?
If you need help installing and troubleshooting a business critical solution, there had better be a company supporting it.