LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-03-2015, 04:13 PM   #1
Miati
Member
 
Registered: Dec 2014
Distribution: Linux Mint 17.*
Posts: 326

Rep: Reputation: 106Reputation: 106
using tmpfs leading to program crashing


I ran into a odd thing running a split command.

I have a tmpfs with 14gb of allocated size. I have 16gb of ram

I have a single 6gb file on HDD which I attempted to split into 200M chunks into my tmpfs
Code:
split ~/Downloads/RAW.tar.bz2.gpg -db 200M image_
Logically, 6gb of split files should not overwhelm a 14gb tmpfs.
The first time I ran it though, I think the whole X display server crashed. It restarted and brought me back.

Of course, I had to try that again.
So I ran it again and this time it killed my firefox process (using ~1gb of ram), then the time after that it killed banshee.

I thought tmpfs prevented you from using more space then possible & crashing other programs?

Last edited by Miati; 01-03-2015 at 04:20 PM.
 
Old 01-03-2015, 09:02 PM   #2
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
tmpfs is in memory.

You fill memory when it fills. If the system needs memory and can't get it - programs get aborted.

That is the problem with tmpfs. It really wasn't supposed to be used for /tmp. Originally, it was for lock files, small credential files that get deleted on logout; it works well for pid files recording service information.. It shouldn't be used for user storage though as it is trivial to create a denial of service condition.

If you have given tmpfs 14GB (out of 16GB), then you only have 2GB for running the system. You can actually take the size of tmpfs (actual usage) and subtract it from your memory size, and that is all you have...

You can try adding more swap - but your system will slow down a lot.
 
Old 01-04-2015, 01:23 AM   #3
veerain
Senior Member
 
Registered: Mar 2005
Location: Earth bound to Helios
Distribution: Custom
Posts: 2,524

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
How you have allocated 14 gb to tmpfs. Are you using swap space. As tmpfs can be swapped to swap space.
 
Old 01-04-2015, 05:23 PM   #4
Miati
Member
 
Registered: Dec 2014
Distribution: Linux Mint 17.*
Posts: 326

Original Poster
Rep: Reputation: 106Reputation: 106
Quote:
Originally Posted by veerain View Post
Are you using swap space. As tmpfs can be swapped to swap space.
Actually no, I turned off swap space. Thinking about it, that likely was a contributing factor to it crashing.
 
Old 01-04-2015, 05:25 PM   #5
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Quote:
Originally Posted by Miati View Post
Actually no, I turned off swap space. Thinking about it, that likely was a contributing factor to it crashing.
An even larger reason to not use tmpfs.

As I said, it makes a local DoS trivial.
 
Old 01-05-2015, 12:41 AM   #6
veerain
Senior Member
 
Registered: Mar 2005
Location: Earth bound to Helios
Distribution: Custom
Posts: 2,524

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
Using tmpfs is fine. You can limit it's size at mount and remount time by passing -o size=xx% or -o size=XXM. ( xx B/KB/MB/GB ).

More info in man pages of mount.

Code:
man mount
 
Old 01-05-2015, 06:19 AM   #7
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Using tmpfs is fine for some things...

But it is entirely possible to kill a system with it.

ANY filesystem that users can write to is not exactly controllable. tmpfs doesn't support quotas (it can't as you would have to reload them on every boot).

And the way RH/Fedora uses them permits a DOS every time. User files in /run allows a user to fill that filesystem - which is used for every login. Fill it, no logins. Not even root. System services cannot be restarted.
 
Old 01-05-2015, 06:52 AM   #8
Teufel
Member
 
Registered: Apr 2012
Distribution: Gentoo
Posts: 510

Rep: Reputation: 120Reputation: 120
Quote:
Originally Posted by Miati
I have a tmpfs with 14gb of allocated size. I have 16gb of ram

I have a single 6gb file on HDD which I attempted to split into 200M chunks into my tmpfs
Split command might require a lot of RAM to deal with your huge file. In fact it operates in RAM, not in tmpfs. Tmpfs is just a storage. And seems your 2Gb of RAM too small for entire the system plus split (which tries to deal with 6Gb file inside 2Gb of RAM).
I think you need to create tmpfs as little as possible to fit your file, something like 6Gb or a bit large. But it should be enough to fit your file. This case you will have almost 8Gb of RAM for your system. And split will be much more comfortable with 8Gb of RAM than with 2 Gb.

Last edited by Teufel; 01-05-2015 at 07:08 AM.
 
Old 01-05-2015, 08:16 AM   #9
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
split by itself uses very little memory.

But it will induce a large buffer pressure, that in competition with normal system usage could push the system into OOM with only 2GB available. (first source I found was from BSD, in the answers to a question of getting the source: https://groups.google.com/forum/#!to...sc/7SwF9wEc21Q)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Programs seldom crashing trebek Linux - Software 3 05-12-2008 02:39 PM
Image programs crashing DAChristen29 Linux - Software 2 11-15-2005 02:26 AM
Programs keep crashing markb658 Red Hat 4 06-02-2005 06:02 AM
Inconsistent crashing of tv programs jimdaworm Linux - Software 1 09-28-2003 05:19 PM
Programs Crashing. rvijay Linux - Newbie 5 09-03-2003 09:52 PM


All times are GMT -5. The time now is 03:59 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration