In comparison, everything for the typical Windows user is usually in an .exe or with mac a .dmg
I'm just wonder what the benefits are with these various methods. I'm sure some of it has to do with the history of Linux or that it's based on Unix.
Exactly, it's evolutionary. Traditionally, *nix systems have their software built/installed from source. The source can be archived in a tar and compressed, ie. bz2, gz. Package managers for different distros are ways in which those distros have tried to make that job less necessary or easier. rpms and debs are not executable programs like "exes" in Windows, they're closer to "msi" installer programs.
Both Windows and MacOS are the products of single companies, hence the apparently more unitary approach. But it isn't all exes with Windows. There are the installer msi and dll's one has to download manually occasionally.
Benefits? As the previous poster mentioned, it gives you more freedom.
Hope that clears things up without actually being informative.