Is there a setting someplace to change this?

When using my touchpad on my Dell C610 laptop (not a problem with the mouse) I experience an odd problem. When having more than one window open at the same time (for me, usual) Fedora will hop between windows by my just moving the mouse. When I'm in Firefox and move the mouse often the page I'm on will just decide to backup to the previous page. Sometimes just moving the mouse will take me to another desktop and show some window there. This is just a bit annoying. Only happens with the touch pad. I hate needing to have only one application open at a time when I don't have a mouse attached. This can't be normal....

Basically, your apps are detecting the mouse scrolling and are configured to bind an event to it.

For your firefox issue, go to about:config in the firefox address bar, then find the following two config variables and adjust them as shown here:

mousewheel.horizscroll.withnokey.action (change it to 0 to turn off scrolling)
mousewheel.horizscroll.withnokey.sysnumlines (change it to “true”)

For the desktop changing problem, I presume you're using compiz/compiz fusion - there is settings in there to alter the behaviour of mouse scrolling - I would guess that you currently have it set to change desktop when the mouse is scrolled sideways.