Many CPUs are better at addressing "memory" than they are at addressing "I/O Ports." There are just more and more-flexible instructions available for memory access.
So, hardware designers build their cards to recognize "read" and "write" requests to certain memory-addresses as being directed to certain hardware-latches on their cards. That is, when you "read address $XYZ," you get the contents of a particular hardware circuit. Likewise, a "write" sets the values of a bank of certain switches.
The operating system arranges for those memory addresses to be protected such that only the kernel can read and/or write to them.
The practice is so widespread and so useful that some microprocessor designs don't even have I/O-port instructions. The designers judged that they simply didn't need them.