ZFS rec0rdsize parameter, anyone have experience?

postcd · 03-11-2021, 12:08 PM

Hello,
i was wondering which ZFS rec0rdsize to set if my average disk read size operation (zpool iostat -r) is from 512 up to 128k while the mean is around 64k.
I am wondering if i should rather do not set it and let Z.F.S detect the size on its own or set it really to that mean value.

TB0ne · 03-11-2021, 12:24 PM

Quote:

Originally Posted by postcd

Hello,
i was wondering which ZFS rec0rdsize to set if my average disk read size operation (zpool iostat -r) is from 512 up to 128k while the mean is around 64k. I am wondering if i should rather do not set it and let Z.F.S detect the size on its own or set it really to that mean value.

Again, you provide no details that would let anyone give a meaningful answer.

Most everything about ZFS in regards to write-amplification takes into account the media being written to, the controller, physical connection, mean size of the files, types of data, etc., and none of that is present here. Not even the version/distro of Linux. Which leaves most things up to you to decide.

https://www.usenix.org/system/files/...16_09_jude.pdf
https://www.reddit.com/r/zfs/comment...ecordsize_faq/
https://www.reddit.com/r/zfs/comment...really_better/

Again, you can adjust your own system on the fly and tune it for what's best for your system/data/needs, and find out for yourself. You asked for 'experience'...mine is, I made adjustments to my system until I found what worked for me.

postcd · 03-12-2021, 03:48 AM

Arnold, i do not think it is correct. I do not understand most of the things you have written (abstract and technical). Only thing i understand that you think that r€cordsize has nearly no impact on reads. Why i think this is not true? Because if there is 1GB file and the r€cordisze is set to 4K then the drive would have to do so many transactions per second to read the file and on HDDs the tps is scarce. Explain why i am wrong. On another hand, if i set RS to 1M and there is 1K files that needs to be read, then i would guess it would cause big problems utilizing HDDs bandwidth internal as it needs to read whole 1M "block" just to get small file?)? more about this there when you search "What if i"

TB0ne · 03-12-2021, 09:02 AM

Quote:

Originally Posted by postcd

Arnold, i do not think it is correct. I do not understand most of the things you have written (abstract and technical). Only thing i understand that you think that recordsize has nearly no impact on reads. Why i think this is not true? Because if there is 1GB file and the recordisze is set to 4K then the drive would have to do so many transactions per second to read the file and on HDDs the tps is scarce. Explain why i am wrong. On another hand, if i set RS to 1M and there is 1K files that needs to be read, then i would guess it would cause big problems utilizing HDDs bandwidth internal as it needs to read whole 1M "block" just to get small file?)? more about this there when you search "What if i"

If you don't understand what you were told, why would you think it's incorrect??? And why ask if you think you already know the answer?? If you don't understand the technical papers and explanation, how can anyone explain things?

Again: find out for yourself. Since you are STILL not providing any details that would let anyone even guess at a suggestion. And since you can easily adjust these parameters and run tests on your own system, for your own environment and get the best results for you, why aren't you?

postcd · 03-13-2021, 03:13 AM

It seems no one has idea based on mentioned stats.

onebuck · 03-14-2021, 08:33 AM

Members should consider placing the other member on their ignore list via LQ UserCP when this type of condition exists. Since both are not helping within the thread(s) by this type of posting between each other. To continually report each other helps no one. This type of communication between each other is not allowed here at LQ because each of you agreed to abide by the LQ Rules

Quote:

Personal attacks on others will not be tolerated.

Do not post if you do not have anything constructive to say in the post.
When posting in an existing thread, ensure that what you're posting is on-topic and relevant to the thread. If the content of your post will interfere with the current discussion, you should start a new thread.
Challenge others' points of view and opinions, but do so respectfully and thoughtfully ... without insult and personal attack. Differing opinions is one of the things that make this site great.

Along with reading LQ Rules members can find helpful information at https://www.linuxquestions.org/quest...hp?faq=welcome

If each of you continue to act in this manner will result in infractions to both!

postcd · 03-16-2021, 08:31 AM

Since no one knows the answer so far, i have left default value 128k. I have read that the ZF$ should automatically put smaller files in smaller blocks than that. So hopefully i set it right, i am guessing no one knows how to discover if it is right from zf$ stats so i will be hoping.