You can list the contents of the tarball with "tar --list -jf sourceryvsipl++-1.3-linux-x86.tar.bz2"
Consider piping the output to "less" and examining the contents. Make sure that none of the files look like something that you need to keep. Look at how the directories are listed. If they don't begin with a slash, then you need to cd to '/' before removing the files.
You can pipe the output of the listing to xargs to delete the files in the list. However, if any of the filenames contain spaces or other special characters, it would be better to use the NULL character as the line separator in xargs.
tar --list -jf sourceryvsipl++-1.3-linux-x86.tar.bz2 | tr '\n' '\000' | xargs -0 rm -v
It would probably have been better if you used /usr/local/ as the base of the installation as per the installation instructions. That is the proper place to install non-distro packages, and you are also guaranteed that the distro won't touch these files.
Sometimes if there is a large number of files in various locations, using /opt/ is a good choice. For example, You might use /opt/kde4/ if you were to install kde4 from source.