the performance gain surely depends on the applications you use. Some will be faster, some will stay as fast / slow as they are. I myself have a Pentium3, and therefore compiled almost everything with CFLAGS="-O3 -march=pentium3 -mcpu=pentium3"
"Almost" because I did not get some packages to compile when optimizations were set, especially glibc was problematic.
When the LFS book says "don't use optimizations here" you'll save yourself some trouble and time if you just follow that hint...