I think you're going to need pvresize + shrink option http://linux.die.net/man/8/pvresize
pvmove allows you to move the allocated physical extents
which I'm pretty sure means all of the allocated extents, used or not ... and sdf1 is 500GB.
These links may also be useful if you haven't already got them
However(!) I've never done what you are trying to do, so you may want to wait for someone else to comment before doing anything.
Ideally of course, find spare box that you can play with, create tiny PVs to test and see what happens without a pvresize.