You don't have the "mysql" database, which is used to store users, passwords, rights etc.
I don't know if the installation using RPMs should have created it, but since it does not exist you must create it prior running mysqld for the first time. In the source and binary distributions there is a script named "mysql_install_db" which creates the initial mysql database. Search for it in your installation location and run it.
Another thing you have to take care is that perhaps old_data uses isam tables while mysql 4.x uses myisam. If that's the case you must run another script named "mysql_convert_table_format" to convert isam to myisam. Take a look here
for more details about upgrading from 3.x to 4.x.