Two ways of doing this:
1. Copy your whole data dir in /usr/lib (OR local)/mysql by hand or script (tarballing it or cp or mv or desktop mgr Copy / Paste)
2. or use mysqldump to export all DBs into INSERT SQL files (with data of course) like mysqldump --opt [DBname] > [/file/path/][fileName].sql
As for using dump for ALL DBs, I haven't tried it but hey, why not?
I'd try it after tarballing the data dir JIC.