The “drivers” are by chipset and will are included in the vanilla kernel, so you can get the sources from there. For example,
here is the driver source for the popular FTDI chipset.
As TB0ne mentioned, if all you need is for the device to work as a serial port, and are not interested in the internals, you will normally just plug it in and udev will take care of the rest.