Regards deleting them, when you would no longer re-install from them then there's no point in keeping them (examples: change of OS or you would install another software or a later version).
Regards keeping them securely, they are normally replaceable in the same way you got them so having them is only a convenience. You can balance the cost of keeping them securely (HDD and backup space usage, administration) against the benefits of having them (time saved if you need to re-install) taking into account the probability of loosing them and re-installing. It could be called "probabilistic cost/benefit analysis"
FWIW I keep them in dedicated directories (/home/c/d/Repository/Slackware <version or noarch> packages/) and those directories are routinely backed up with no special effort. When the file system is short of space I delete obsolete files in those directories.