There are many different flavors of ARM CPUs. Each has its own subtle differences. Some use different types of floating point processors, some have slightly different instruction sets, and all of the on-chip peripherals are arranged differently, and controlled differently. If you build a cross toolchain for an ARM architecture host, you will quickly see how the number of permutations grows. Much the same configuring a kernel.