I ended up buying a fitlet PC
http://www.amazon.com/gp/product/B00...A3J2VUK9571N8X
It has linux support listed, this model includes three aditional ethernet ports, and it supports adding up to 8gb of memory and a m2 sata. Lots of USB, and it is still small and fanless. I ordered it with an 8gb SODIMM and a 250GB Samsung 850 EVO SSD, and it should work great.
I appreciate your thought and suggestions. For those who are curious I will probably post a software build log later if I make something interesting. Not all sure how I want to do it yet, but I am thinking about spinning up some VMs for web servers and making it into a bad ass router

.
Thanks again.