The command doesn't install a package. It justs expands the file. Most tarballs will expand into a directory, so you just need to delete that directory and all files in it.
If you have a tar file with a number of files that were extracted in the current directory, you can list the files in the original tarball with "tar --list -jf tar1.tar.bz2". The file list can be used to delete the files:
tar --list -jf tar1.tar.bz2 | tr '\n' '\0' | xargs -0 rm -v
The "tr" command will replace returns with the null character. The -0 option of xargs will use the null character to separate arguments. This is needed if the file names contain white space.