New\ Folder
certainly should work. So should
'New Folder'
So should
N<tab>
You didn't literally type what you said, right? You did 'cd' to it or 'ls' it or something?
-- Oh, and you couldn't just rename the file without escaping it in some form or other just as you would for a cd or ls, else you'd be trying to rename two folders that don't exist - 'New' and 'Folder'.
-- Look what you got me saying. Rename two 'directories'. 'Folder' is a bad metaphor that leads to inaccurate mental models of what's going on.