There isn't a meaningful answer to this question. Having looked at a fair number of benchmarking tests, I don't think that you can truthfully say that 'Distro A' is faster than 'Distro B'.

There are differences, but some of those come down to which version of the kernel they use. These won't (shouldn't, usually) be big differences, but whichever gets the faster kernel first (or, slower kernel last) will usually be caught up by the other one at the next release

Some of the differences will be down to default configuration (disk subsystem and things like the task queueing), and you can put that 'right' by adopting the faster configuration. Well, except for the cases in which the faster configuration is in some way the riskier config and you may not want to do that.

Having enough memory is A Good Thing (TM). When you are just at the limit with whatever it is that you use, a set of compilation options that make your software more compact will help. Other times it will do the opposite of helping, so that is going to make it difficult to decide which is faster.

In general, smaller and slicker graphical user interfaces may help, when you are at the edge. And that's fine, but having enough memory makes a part of this problem go away, and not using 'bloaty' packages that put you close to the edge is probably better. And, usually, that's a matter of selecting the right packages and you can do that with any Linux, if you put the effort into it. Some will have a better (more appropriate) choice of defaults, but if you are going to let the fact that some packages are default deflect you. you weren't taking it seriously. (Some times the 'bloaty' packages do more for you and in those cases, bloaty can be faster, because getting the computer to do it is usually faster than doing it manually. Depends on your use case.

Hmm, you CAN make any distro faster by changing apps and reconfiguring kernel etc, but some distros have certain versions of certain SW. And when you do a lot of modifications, you'll end up with LFC (Linux From Scratch). You have very little common with the original distro, so you can't really talk about the original diströ, but what you CAN talk about, is compatibility problems!

Most distro's can be stripped. It isn't that hard to re-compile the kernel only to match your system but on each update you'd have to recompile. Some distro even has a way to do that each time.

To speed up in other ways one might look at bottle necks. A big on on that system is the drive. It may be that making some compressed drive would speed it up. Something as simple as trying ext2 filesystem may even improve speeds.

You should check filesystem benchmarks if you want to know what filesystem is fastest. Make sure to check HDD versus SSD, because they are not the same. Last benchmark I checked, btrfs was fastest for HDD (with ext4 second fastest), and JFS was fastest for SSD. However, I don't trust btrfs ATM.

There have been quite a lot of good ideas so far in this thread. And they all are valid points. Keep a low footprint and it'll fly.
But then everyone seems to just be pushing his own choice. I'm using Aptosid because I'm a professional (development and administration) and still like my system semi-current. But for speed I'd check out Gentoo. Of course, the time you save because of the speed optimizations is likely to be lesser than the time you spend optimizing. But in my opinion an OS you can compile specifically for your own hardware has the best potential. But then I don't know about availability of proprietary (usually faster) e.g. nvidia/radeon drivers.