First of all you need to know enough of digital electronics to be able to read the hardware documentation - you need to understand how to use the "control registers" of a device as well as how to read/write data to that device. You also need to be comfortable reading timing diagrams for the hardware and be able to spot any timing issues which may come up.
The simplest of projects would be to write a small driver for your UART (serial port hardware). Identify the exact hardware part, get the documentation, and start writing code for it. Your code can set a fixed bit rate, no hardware handshaking, no control lines - make it simple so you can test it by connecting to another serial port and running 'minicom'. The UARTs are one of the simplest devices and if you screw up the driver you don't do yourself any harm (if you tried your hand at a PS/2 serial driver and screwed up, you suddenly don't have a keyboard or mouse and may be forced to do a hard boot).
For information on device drivers, have a look at Linux Device Drivers 3rd edition (Alessandro Rubini, Jonathan Corbet, Greg Kroah-Hartman) - the electronic version is freely available.
Now anything to do with UARTS actually becomes a horrible mess in UNIX because these were traditionally used to support serial terminals. So if you have a look at the serial port code in Linux you will see that there is the base code which enables interrupts, and reads and writes data. However, data is never directly written to (or read from) the device from user space - data read from the device is passed onto the "line discipline" (a filter) for processing. When the user writes data, that data is also passed onto the line discipline for processing and the line discipline then passes the data on to the serial code which actually writes it to the UART.
Anyway, writing a UART driver will give you the simplest possible hardware driver I can think of and it will help you get familiar with the system. After that, what you study depends on what type of driver you want to make. If it is anything which goes onto the PCI bus then you need to study the parts of the Linux device driver system that relate to PCI (how they are discovered, how to request resources, etc etc). If it is a USB device then you need to look at the USB driver code structure. USB involves a protocol layer as well as the hardware driver for the USB controller chips. When you plug in ANY USB device, certain messages are sent along the line as part of the protocol. You basically have to learn how to interact with the protocol layer; your driver will have to work through that layer to discover and initialize your device and to process all incoming/outgoing messages after that. When dealing with a USB device, it is possible that you never write any code to initialize hardware. When dealing with PCI devices, you will need to set up the specific PCI controller which your device uses (the controller will be partially set up by firmware/ROM, but you need to do more setting up such as telling the controller what interrupts to use etc).