How to force disk caching even if software uses O_DIRECT?

leniviy · 04-16-2016, 01:39 AM

Hi.
Some programs disable OS caching by using O_SYNC or O_DIRECT flags, because they think they can do better caching than OS. Sometimes they're wrong and become very slow.

Is it possible to create a stack of virtual block devices that silently ignores O_DIRECT at some level? Or maybe some filesystems can be tuned to ignore it? I don't care about crashes or power failures, it's not a production environment.

malekmustaq · 04-16-2016, 04:12 AM

Before opting for a stack of virtual block here is something that may improve the needed performance.

syg00 · 04-16-2016, 05:11 AM

I would guess the OP is attacking the symptom rather than the problem.
Let's see some evidence.

leniviy · 04-16-2016, 05:35 AM

Quote:

Originally Posted by malekmustaq

Before opting for a stack of virtual block here is something that may improve the needed performance.

just read that article. I can't change i/o scheduler, because the only available value is "none" (and it's correct, because it's a virtual machine with virtio storage)

Code:

/sys/block/vda/queue/scheduler:none
/sys/block/vdb/queue/scheduler:none

Other advices are for those who write software; mine is already written.

smallpond · 04-16-2016, 06:12 AM

If you have plenty of memory you can mount a tmpfs, write there and copy to disk manually or with a background rsync process.

leniviy · 04-16-2016, 11:14 AM

Quote:

Originally Posted by smallpond

If you have plenty of memory you can mount a tmpfs, write there and copy to disk manually or with a background rsync process.

Attempt to open a file in tmpfs with O_DIRECT fails.

smallpond · 04-17-2016, 06:32 AM

I found a thread on LKML where everyone agrees that tmpfs should support O_DIRECT but no change gets made. It points out that the workaround is a loop device.

https://lkml.org/lkml/2007/1/4/55

leniviy · 04-18-2016, 01:14 AM

Quote:

Originally Posted by smallpond

It points out that the workaround is a loop device.

This seems ugly, but it works. It's actually possible to create loop devices not only for regular files, but for other block devices too:

Code:

# mounting /home/db2inst1 with -oloop to ignore O_DIRECT
/dev/mapper/vg_unsafe-db2inst1 /home/db2inst1 auto loop