This is happening in both gvim and vim.
Thanks for the two links, but I've actually already tried that solution, and it doesn't seem to work!
I did however finally figure it out. It turns out that I had added a set of tex macros that doesn't respect the b:Tex_SmartKeyQuote and g:Tex_SmartKeyQuote flags, and infact, doesn't even respect what mode the buffer is in. For reference, this is the macro's info:
" LaTeX filetype plugin
" Language: LaTeX (ft=tex)
" Maintainer: Srinath Avadhanula
" Email: firstname.lastname@example.org
" URL: http://vim.sourceforge.net/scripts/s...p?script_id=93
" Last Change: 2001 Dec 9
Thanks for the help guys.