What distribution are you running?
You just need to install the bootloader to the second drive. Assuming that the second drive is now in the same location as the first. If not you will need to modify the /etc/fstab to reflect the different device ID i.e. hdx (or sdx) of the new drive in addition to the bootloader configuration file.
The file your looking for is actually /boot/grub/menu.lst and not grub.conf. There are a few distros like Redhat that uses /etc/grub.conf file. In this case the menu.lst is a link that points to grub.conf.
grub-install --root-directory=/mnt/x /dev/hda
Where x is the directory where you mounted the root directory.
Assuming your really using grub. If using lilo then its /etc/lilo.conf.