How to force disk caching even if software uses O_DIRECT?
 

Hi.
Some programs disable OS caching by using O_SYNC or O_DIRECT flags, because they think they can do better caching than OS. Sometimes they're wrong and become very slow.

Is it possible to create a stack of virtual block devices that silently ignores O_DIRECT at some level? Or maybe some filesystems can be tuned to ignore it? I don't care about crashes or power failures, it's not a production environment.

Before opting for a stack of virtual block here is something that may improve the needed performance.

I would guess the OP is attacking the symptom rather than the problem.
Let's see some evidence.

just read that article. I can't change i/o scheduler, because the only available value is "none" (and it's correct, because it's a virtual machine with virtio storage)

Other advices are for those who write software; mine is already written.

If you have plenty of memory you can mount a tmpfs, write there and copy to disk manually or with a background rsync process.

Attempt to open a file in tmpfs with O_DIRECT fails.

I found a thread on LKML where everyone agrees that tmpfs should support O_DIRECT but no change gets made. It points out that the workaround is a loop device.

https://lkml.org/lkml/2007/1/4/55

This seems ugly, but it works. It's actually possible to create loop devices not only for regular files, but for other block devices too: