LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Kernel (https://www.linuxquestions.org/questions/linux-kernel-70/)
-   -   How to disable filesystem buffer for a single filesystem? (https://www.linuxquestions.org/questions/linux-kernel-70/how-to-disable-filesystem-buffer-for-a-single-filesystem-4175615343/)

usao 10-09-2017 07:06 PM
How to disable filesystem buffer for a single filesystem?
 
I have a 2TB SSD drive which im writing data to using 'scp'.
The 'scp' command finished but in trynig to umount the drive, I found that I had to run a 'sync' command to flush the buffers.
The disk remained at 100% utilization for almost 1.5 hours while the sync command ran.
Unfortunately, this caused me to miss a window for shipping the drive out.

So, how can I disable the filesystem buffer to prevent this crazylong sync command?
I would much rather have a slower transfer rate but be able to get the drive out of the system on schedule than have faster transfer and miss my delivery.

syg00 10-09-2017 07:26 PM
Short answer - you can't.
How is the SSD connected ?.

usao 10-09-2017 07:45 PM
via USB 2.0, the drive is a Samsung 2tb consumer drive with a usb adapter.
This is for our offsite backups, so pick-up tome is important.

Code:
Device:        rrqm/s  wrqm/s    r/s    w/s    rMB/s    wMB/s avgrq-sz avgqu-sz  await r_await w_await  svctm  %util
sdg              0.00    0.00    0.20  54.80    0.00    27.40  1020.31  141.10 2569.24  601.00 2576.42  18.18 100.00

syg00 10-09-2017 07:52 PM
Thought so - in that case you can do some things to help. Read this. It's old, but still relevant.

usao 10-09-2017 08:16 PM
That's conceivable since the box has 1tb ram...

michaelk 10-09-2017 08:48 PM
I might be over simplifying things but USB 2.0 is 480Mbps. How much data are you writing to the drive? If you are writing 300GB then it will take about 1.5 hours. You can easily calculate the approx time it will take to transfer and write to the drive so you can plan accordingly.

usao 10-09-2017 09:28 PM
Well, I get about 27 MB/s or just over 50% theoretical bandwidth.
Ive been adjusting the dirty_bites downward slowly to avoid stalling the ssh copy.
Got it down to 90GB and heading down to 10GB.

Code:
[root@hp6 vm]# for x in $(seq 800 -1 100); do echo ${x}00000000 > dirty_bytes; sleep 10; done
Seems to be working, the I/O throughput is slower but not stalling, which is important as resuming a scp command is a problem because it doesnt purge copied files and doesnt go in alphabetical order either, so its difficult to restart. Once the files are transferred I can go back and generate a list of files to delete, then swap drives and resume the scp.


All times are GMT -5. The time now is 12:56 PM.