I’m writing a crypto device driver for a packet based engine. This means the DMA is NOT capable of scatter/gather as the engine assumes “packets” in one continu block.

The driver is getting a Source and Destination scatterlist (amongst others).

Simplest would be to sg_to/from_buffer. This comes with a performance penalty and seems not logical since I could use DMA.

Alternatively I could build a new array of physical addresses and lengths based on the two original lists and feed them one by one to the DMa controller. Actually this could be done in “one” go as the engine is capable of getting the next packet by itself. After all the separate segments of the array are completed I could then signal the calling application completed (as opposed to signal ready as soon as the engine sends an interrupt that the packet is done).

Third option could be a combination: dma the separate scatterlist segments into one buffer using offsets and the copy the buffer. (This would give overhead for the copy, but would reduce the amount of DMA required).

Is there any way to tell which would be the best in terms of performance or do I need to do a full benchmark on all three options?

The end goal is go get this driver into the Linux tree, so with that in mind, would there be a preferred way of doing this?

Do I understand correctly that this engine is capable of handling scatterlist but it requires to have complete packets and not packet fragments? What is your use case? Do you want to encrypt network packets or there is other thing to do? The answer will tell what is the approach that would work from the performance side.

Have you considered "flattening" your packets, i.e. copying the smaller parts to the big buffer (if space available, of course) and the DMA the complete buffer?

As for mailine kernel, I think that the best approach would be to find out the best way to use this DMA and then implement it. There are some approaches that won't pass for mainline, but those are extreme cases: for example if you decide to have a active polling thread without sleeping to wait for completions.

For me a scatter/gather dma means: a separate scatterlist for “input” and one for “output”. Those lists don’t have to match. Depending on the operation, the total length should match, but not the individual segments. The input could be e.g. 4 segments and the output only 2 for a given length.

This hardware expects an “input (physical) address”, an “output (physical) address” and a “length”. So if I get two scatterlist via the API call to the crypto driver it is very likely that they are not the same fragmented.

Scatterlist to a linear buffer copy, do the DMA on the buffer. DMA back into a buffer and coy the buffer back into a scatterlist. However this seems to defeat a “zero copy” approach which should be possible using DMA.

As for using it via the XFRM offload API, then it gets a SKB which might be non-linear. This would be solved by a system call to make the SKB linear again. (Possible advice the user to always use linear SKB for the underlying network.)