LXDE is faster, the main reason being that it uses Openbox as its window manager, which is smaller and quicker than XFWM. Its components are also smaller, making them quick as well.
However, be aware that you'll be making a sacrifice for the extra speed. The LXDE components are very minimalistic, meaning you'll have less convenience with those tools. If you're a newer Linux user, you may find that the components are lacking features that you're used to having.
If you're using a fairly recent computer, you probably won't notice the difference between the two. But if you're using a netbook or a very old machine, LXDE is probably your best choice.