It will stay there forever, as any other regular file, until you remove it. If you want to remove all the backup files at once, use find:
Code:
find $HOME -type f -name \*~ -ok rm {} \;
the -ok action just asks you if you want to remove the file. If you trust the result of your find command (and you want to spare typing yes over and over) just use -exec in place of -ok and the trick is done.