Where is the code that writes to SSD in Linux kernel?
 

Using Ubuntu 16.04 with kernel 4.16.7. Also, I have an Intel Optane 900P SSD mounted.

I am trying to find a code in the kernel that does writing to SSD in order to try to optimize it using DMA that is present on a target board running kernel 4.4.84.

I am aware of the NVME driver but was unable to find the code that actually does write to SSD. I would like to ask if I am looking in the right place and, if not, where I should be looking. Also, is any improvement even possible in the existing drivers or are they already optimized?

Thank you in advance.

I think you may need to look in the kernel module code for that device.

It's very much in the NVMe driver. The PCIe part is in drivers/nvme/host/pci.c and the little more generic part in drivers/nvme/host/core.c. The main "write" code is in nvme_queue_rq and consists of creating the command that is submitted to the disk. You won't find traditional DMA here because it's the disk DMA that does the work. All DMA-style parameters are in the SQE. You can also look at nvme_submit_io, it creates the command (SQE in the NVMe language).

All code can be improved. Whether you are competent, or can get your changes accepted is another matter.

The kernel is extremely fluid - and frustrating. Even if you have a valid patch, the underlying code may well change before you even get through testing your code. Then you need to start again. Maybe several times.

And you have to deal with the egos in the kernel dev community. Good luck.
Of course there is nothing stopping you maintaining your own out-of-tree patches.

NVMe is a very fast moving piece of code those days. The code is already pretty robust, but you can always find improvements, especially if you have a specific use-case in mind. And people developing it are very friendly while having high quality requirements.

Hi everyone, thank you so much for your replies, I posted the same question on another forum and got no answers.

The thing is, we are connecting our custom board to SSD using Gen2 x2 PCIe and getting ~380MB/s which is about half the theoretical speed. But it may very well be the problem of the board itself whose CPU is already nearing 100% of load while writing to SSD. This is the reason why we are trying to optimize it with DMA.

Generally, we lack better understanding of the whole process. I suppose that when data arrives to NVMe driver it's too late for any DMA improvements. Only the part while the data is being sent to the NVMe driver for it to submit it with a command can be optimized with the board's DMA, am I right?

We are also trying to change the scheduler that is currently present in the NVMe driver, do you think it will yield some better results and, if you have some advice regarding that, it would be much appreciated.

Thank you again and looking forward to hearing from you.