I doubt you'd ruin the pendrive, but you might corrupt your data.
To safely remove it, you just need to make sure it's unmounted. If you mounted it manually, then you probably know how to unmount it
If your system automatically mounted it for you when plugged in, it probably put an icon on your desktop to access it. In Gnome, you can right-click the icon and choose to unmount it from there. I'm sure KDE has something similar.
Once unmounted: yank it!