Depending on your distribution, you may not have to recompile your kernel. The installation CDs may have a separate package containing a kernel with SMP support. If you fire up your package manager and can browse the list of packages, look for packages that mention "kernel", "SMP". Accessing the second processor might be as simple as installing that package. (I did something similar: replaced a PIII-based m'board with a P4. Installing the SMP kernel was all that I needed to do to use the hyperthreading feature of the CPU.)
If your distribution doesn't include an SMP-aware kernel -- which would be surprising -- then you'll have to read up on compiling the kernel with SMP support. And... you'll have to know about configuring your boot loader to recognize and boot the newly compiled kernel.
Did your kernel compilation adventure result in errors? (You did install the kernel sources and all the headers, etc., etc., right?) Or did the new kernel just not boot? Or what?