provides a good explanation of international text support. It's focused on Debian, but the info can be easily applied elsewhere.
Short form, you need to install scim (or uim, if you prefer) and the scim-anthy Japanese backend (or other backends for the languages you want) and set up some shell variables. Then you can activate the IME with shift+space (Edit: actually, I think crtl+space activates the IME, and shift+space cycles through the available languages). This assumes you know the basics of using an IME, of course.