Please be patient with me here... I'm fairly savvy with Linux, but I'm not completely clear on all the workings of the Internet:

One of my web sites is hosted by a cheap web hosting service. Okay for the price, but I didn't have any control over the system, and the server they had me on was always really slow.

So, I got myself an account elsewhere for a self-managed box (dedicated computer that I SSH into and have root access), which also provides a static IP address. I turned it into a Web server, with the intention of dropping my hosted Web account, and hosting my website on my own box. So lately I've been looking for a domain registrar service so I can transfer the domain away from the hosted account.

I kind of assumed that the "domain registrar" service and the "name server" service were the same thing. But from what I'm reading on these sites, it seems like the domain registrar just points to the name servers.

So, if these sites I've looked into (namecheap, Misk so far) don't include name servers as part of the package, do I need to pay for name servers as a separate package? Or do I need to turn my self-managed box into a name server that points to itself? Or am I misunderstanding something about how all this works?

You can
a) pay someone (like your domain registrar) for DNS hosting, or
b) host DNS yourself, or
c) use a free DNS service like afraid.org

I use (c).

I use dyndns. If what you are doing is small then that would probably suffice. Normally your hosting service has their own DNS servers and you have the option to use theirs or choose you're own as I understand it.