I'm doing the same thing ! After months of research and help from members on this forum, this is what I've got.
A fast dedicated only to audio PC with large GB hard drive, a Pentium 4 in my case, a duo-core might work better but I can't afford a top end duo-core. 1 GB of memory to start with maybe 2 GB in the future, a M-Audio 44 4 track sound card with a M-Audio Buddy, Shure SM57 microphones for voice and instrument.
The three most important software items are the Mandriva 32 bit enhanced multimedia operating system offering close to realtime throughput, an all inclusive ALSA sound package that will control your PC and hardware and Audour recording and production software.
As an introduction to music production on Linux, I recommend the following packages that operate from the command line, SOX, LAME and Ecasound. These packages will get you started when you have the hardware.
Setting the machine up to work exclusively with the free ALSA software components has been one of the most gratifying experiences that I have used to create a music PC, it's like having a tube based amplifier controlling the quality of the sound coming out !
I'm still building the PC in my spare time for my daughter who is a music major in college.