You might try memtest86 on the machine. Having bad RAM or running out of it is one thing that will crash your system, regardless of the operating system. And running the system too hot is another. If you're running an older distro, you might try a newer one, but more applicable if the hardware was new-ish. And patch your system if you use 3rd party things like flash. Yet another zero day patch from adobe, there's been at least five that I've been aware of and patched just this year and it's only mid-march.
If you suspect the swap, you might try a fresh install on another usb storage option and see if the issue persists. Drives and flash cards don't last forever. At a minimum you could have a 2nd device, say a card reader and SDHC card, for just swap and see if the problem goes away or is less severe. If the usb drive is usb powered you might try a powered usb hub as laptop usb ports tend to be underpowered.