LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   using tmpfs leading to program crashing (https://www.linuxquestions.org/questions/linux-newbie-8/using-tmpfs-leading-to-program-crashing-4175529947/)

Miati 01-03-2015 03:13 PM

using tmpfs leading to program crashing
 
I ran into a odd thing running a split command.

I have a tmpfs with 14gb of allocated size. I have 16gb of ram

I have a single 6gb file on HDD which I attempted to split into 200M chunks into my tmpfs
Code:

split ~/Downloads/RAW.tar.bz2.gpg -db 200M image_
Logically, 6gb of split files should not overwhelm a 14gb tmpfs.
The first time I ran it though, I think the whole X display server crashed. It restarted and brought me back.

Of course, I had to try that again.
So I ran it again and this time it killed my firefox process (using ~1gb of ram), then the time after that it killed banshee.

I thought tmpfs prevented you from using more space then possible & crashing other programs?

jpollard 01-03-2015 08:02 PM

tmpfs is in memory.

You fill memory when it fills. If the system needs memory and can't get it - programs get aborted.

That is the problem with tmpfs. It really wasn't supposed to be used for /tmp. Originally, it was for lock files, small credential files that get deleted on logout; it works well for pid files recording service information.. It shouldn't be used for user storage though as it is trivial to create a denial of service condition.

If you have given tmpfs 14GB (out of 16GB), then you only have 2GB for running the system. You can actually take the size of tmpfs (actual usage) and subtract it from your memory size, and that is all you have...

You can try adding more swap - but your system will slow down a lot.

veerain 01-04-2015 12:23 AM

How you have allocated 14 gb to tmpfs. Are you using swap space. As tmpfs can be swapped to swap space.

Miati 01-04-2015 04:23 PM

Quote:

Originally Posted by veerain (Post 5295082)
Are you using swap space. As tmpfs can be swapped to swap space.

Actually no, I turned off swap space. Thinking about it, that likely was a contributing factor to it crashing.

jpollard 01-04-2015 04:25 PM

Quote:

Originally Posted by Miati (Post 5295365)
Actually no, I turned off swap space. Thinking about it, that likely was a contributing factor to it crashing.

An even larger reason to not use tmpfs.

As I said, it makes a local DoS trivial.

veerain 01-04-2015 11:41 PM

Using tmpfs is fine. You can limit it's size at mount and remount time by passing -o size=xx% or -o size=XXM. ( xx B/KB/MB/GB ).

More info in man pages of mount.

Code:

man mount

jpollard 01-05-2015 05:19 AM

Using tmpfs is fine for some things...

But it is entirely possible to kill a system with it.

ANY filesystem that users can write to is not exactly controllable. tmpfs doesn't support quotas (it can't as you would have to reload them on every boot).

And the way RH/Fedora uses them permits a DOS every time. User files in /run allows a user to fill that filesystem - which is used for every login. Fill it, no logins. Not even root. System services cannot be restarted.

Teufel 01-05-2015 05:52 AM

Quote:

Originally Posted by Miati
I have a tmpfs with 14gb of allocated size. I have 16gb of ram

I have a single 6gb file on HDD which I attempted to split into 200M chunks into my tmpfs

Split command might require a lot of RAM to deal with your huge file. In fact it operates in RAM, not in tmpfs. Tmpfs is just a storage. And seems your 2Gb of RAM too small for entire the system plus split (which tries to deal with 6Gb file inside 2Gb of RAM).
I think you need to create tmpfs as little as possible to fit your file, something like 6Gb or a bit large. But it should be enough to fit your file. This case you will have almost 8Gb of RAM for your system. And split will be much more comfortable with 8Gb of RAM than with 2 Gb.

jpollard 01-05-2015 07:16 AM

split by itself uses very little memory.

But it will induce a large buffer pressure, that in competition with normal system usage could push the system into OOM with only 2GB available. (first source I found was from BSD, in the answers to a question of getting the source: https://groups.google.com/forum/#!to...sc/7SwF9wEc21Q)


All times are GMT -5. The time now is 08:41 PM.