That's not quite what I imagined. I think this post does a good job of explaining what I was thinking of-
http://brainstorm.ubuntu.com/item/16154/
"At the moment it is impossible to use rhythmbox in case of non-tagged or purely tagged library. However, sometimes this kind of libraries are well-organized in case of folder structure. It will be a huge improvement to add a kind of a file browser (tree view) to the side panel like Amarok and Exaile have. "