It's unusually high if you're not doing anything.
I'm currently running Firefox and Thunderbird, and my memory usage is at about 160MB.
First of all, which memory stats are you looking at? Are you looking at everything, including cache? If so, ignore the cache amounts. The cache value increases, at least in my experience, over time until it pretty much fills up all available RAM. but it's still available because when the system needs it, it takes it. At the moment, my 'in use' memory may be 160 MB, but if you add the amount currently used as cache, it's more like 380MB. That amount will increase over time as I do more stuff.
Second, if you're not including cache values, you'll want to see what services you are running, so you can disable the ones you don't need. In FC4, go to Desktop --> System Settings --> Server Settings --> Services, and uncheck the items you don't need. Save the changes and that should help.