/dev/shm

lopid · 05-29-2011, 09:33 AM

I've never seen anything using /dev/shm. Is there any real point of it? Isn't linux optimal enough in keeping recent files in memory, that it negates the necessity of /dev/shm?

andrewthomas · 05-29-2011, 09:41 AM

google-chrome / chromium uses it.

Woodsman · 05-29-2011, 03:04 PM

/dev/shm is just another branch in the file system. Useless unless something is stored there.

The name shm is an acronym for shared memory. Typically /dev/shm is used with the tmpfs file system. Typically a tmpfs location is created and mounted in /etc/fstab.

By design, tmpfs uses only up to 1/2 of the available RAM, but that can be modified by mount options. Typical usage is far less than 1/2.

Anything stored in tmpfs is lost upon a reboot or shutdown, which for many people is a convenient way of controlling garbage files from accumulating.

The primary benefit of using tmpfs is speed. Accessing files stored in RAM is faster than files stored on a hard drive.

tmpfs can be of more use when the environment variables $TMP, $TEMP, and $TMPDIR are assigned to a tmpfs location.

Once in a blue moon that might not be helpful such as when building packages that require an abnormal amount of build space. The solution then is to ensure the build script does not use tmpfs.

The Linux kernel performs a lot of caching in memory, but to my knowledge tmpfs is not used for that caching.

Others can write plenty more about using tmpfs and the advantages and disadvantages.

Opinions will vary about the usefulness of tmpfs.

rg3 · 05-29-2011, 04:47 PM

Any program linked with a recent version of glibc that uses the POSIX shared memory API (shm_open and related functions) will use /dev/shm. Those programs are not very common but, as you may not be sure which ones could need it, /dev/shm should be configured properly. Here's a sample program:

Code:

#include <assert.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

#define NAME    ("/linuxquestions.org")

int main()
{
        int fd;

        assert((fd = shm_open(NAME, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR)) >= 0);
        sleep(30);
        assert(close(fd) == 0);
        assert(shm_unlink(NAME) == 0);

        return 0;
}

Save that program to a file named posix_shm.c, and build it with the following command:

Code:

gcc -o posix_shm posix_shm.c -lrt

Then, you can try to run it with the command:

Code:

./posix_shm

If the program runs successfully, no messages will be printed on screen and the program will create a shared memory segment for 30 seconds, and then proceed to delete it. In those 30 seconds, if you list the contents of /dev/shm, you will probably see a file in there called "linuxquestions.org".

JokerBoy · 05-29-2011, 08:44 PM

edit: not an offence, it was just an advice.

bgeddy · 05-29-2011, 08:58 PM

JokerBoy - I'm not sure whether you meant it or not but I find your reply highly offensive to a poster who is simply answering the OP's question. It would be much better not to make such initial comments as yours. Your following advice may be good but the initial remark was uncalled for. If such offence was not your intention please consider not using such comments.

lopid · 05-30-2011, 08:40 AM

Thanks, guys. I know what /dev/shm is, I've just never seen anything use it (I have gkrellm monitoring it). I still don't see the advantage of it over the kernel's default caching. If programs are wanting to use it for files that they want to store for longer than the default file cache will allow, well, those files will be held in memory for that time if using /dev/shm, and that's probably not a great idea unless you have oodles of RAM...

speck · 05-30-2011, 02:56 PM

I use /dev/shm for my Firefox profile. On system bootup I copy my FF profile from disk to /dev/shm and on shutdown I copy it back to disk (taking scheduled rsync snapshots in between as well).

Woodsman · 05-30-2011, 03:00 PM

Quote:

I still don't see the advantage of it

Caching is not the same as needing a place to store temporary files. Keeping $TMP clean (the default is /tmp) and compiling packages are two advantages. I assign $TMP, $TEMP, and $TMPDIR to tmpfs. Because of the $TMP assignment to tmpfs, all of my package building takes place in that location. Much faster than using /tmp on a hard drive. As I shut down my system daily I need not worry about keeping /tmp clean. I add a cleanup command at the end of my build scripts to recover my tmpfs space when the package is finished, but most people don't. By using tmpfs they keep /tmp clean.

Working in tmpfs (RAM) is much faster than using a hard drive lovation like /tmp.

Quote:

that's probably not a great idea unless you have oodles of RAM

The tmpfs file system is limited to one-half of installed RAM. That is limited --- not will use or reserve. Right now on my 4GB system I see only 32K of a possible 2 GB of RAM being used by tmpfs. Even when I build packages and the source tarballs and building takes place in tmpfs, I see only MBs of RAM being used in most cases.

I think a general principle is using tmpfs is limited with less than 1 GB of RAM.

GazL · 05-31-2011, 03:37 AM

/dev/shm is specifically for storage of POSIX shared memory objects as rg3 pointed out. Though you may get away with using it for other things as a number of people have described above, it's a misuse and you really shouldn't be putting other stuff in there .

If you want to put your /tmp, $TMP or any other directory on a tmpfs, then create one specifically for that purpose and leave /dev/shm for what it was intended for.

lopid · 05-31-2011, 04:57 AM

I'll do the correct thing and not set $TEMP, etc. to it. I'll also try to stop monitoring it on the desktop, as seeing it unused all the time is a little annoying :-)

Woodsman · 05-31-2011, 03:31 PM

Quote:

/dev/shm is specifically for storage of POSIX shared memory objects as rg3 pointed out. Though you may get away with using it for other things as a number of people have described above, it's a misuse and you really shouldn't be putting other stuff in there .

If you want to put your /tmp, $TMP or any other directory on a tmpfs, then create one specifically for that purpose and leave /dev/shm for what it was intended for.

I read an article at IBM Developer Works addressing the topic of tmpfs. The author wrote:

This is because you need to have kernel tmpfs support in order to use POSIX shared memory. System V shared memory will work without tmpfs in your kernel, however. Note that you do not need a tmpfs filesystem to be mounted for POSIX shared memory to work; you simply need the support in your kernel. POSIX shared memory isn't used too much right now, but this situation will likely change as time goes on.
....
This time, instead of mounting our new tmpfs filesystem at /mnt/tmpfs, we created it at /dev/shm, which is a directory that happens to be the "official" mountpoint for a tmpfs filesystem.

The author of /usr/src/linux-2.6.33.4/Documentation/filesystems/tmpfs.txt wrote the following:

glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for POSIX shared memory (shm_open, shm_unlink).
...
Some people (including me) find it very convenient to mount it e.g. on /tmp and /var/tmp and have a big swap partition. And now loop mounts of tmpfs files do work, so mkinitrd shipped by most distributions should succeed with a tmpfs /tmp.

That implies to me that /dev/shm must be mounted for any POSIX compliant app that expects to use shared memory. Neither author wrote that other apps cannot use /dev/shm.

Both authors imply that more than one tmpfs mount point is acceptable. I believe the Debian folks have two. The IBM author describes that rather than create distinctive mount points, to consider mounting additional tmpfs locations under /dev/shm.

Browsing the web revealed that people are assigning tmpfs to many different places. Besides the official location at /dev/shm, common additional tmpfs mount points are /tmp, /var/run, and /var/lock.

One thing certain is any file or data that needs to be persistent across reboots should not be stored in tmpfs.

You could be right, but I would be grateful to read something authoritative that explicitly says not to use /dev/shm.

Thanks much.

rg3 · 05-31-2011, 04:49 PM

Woodsman, I get the impression you missed the point, but I'm not sure. You can mount tmpfs file systems in many places in the tree. The advice here is not to pollute the /dev/shm namespace with more files when applications using POSIX shared memory may want to create named shared memory segments, which means a file has to be created under that directory. If you want to use a tmpfs for other purposes in other mount points, that's not a problem at all per se.

Woodsman · 05-31-2011, 06:14 PM

Perhaps I am missing the point.

What is being "polluted"? Creating additional tmpfs mount points does not alleviate problems with consuming RAM because all tmpfs mount points reside in RAM. I don't see a difference with filling /dev/shm with files or in a dozen different tmpfs places because the same amount of RAM is used.

I'm curious when problems might arise.

As I mentioned, the only criterion I have read is that a /dev/shm mount point must exist for those apps expecting to use tmpfs. I have read nothing about not using that same mount point for other apps or tasks.

I searched for anything that defined distinctions or requirements. I found nothing authoritative but that could be a result of my search patterns.

rg3 · 06-01-2011, 12:18 AM

It's not about RAM usage but about available file names and mixing regular user-created files with glibc-created special purpose files. Every file you create in /dev/shm for other purposes is one less file name available for a shared memory segment, and it doesn't belong to a shared memory segment.

The advice is to separate the two concepts. Leave /dev/shm clean and only use it for shared memory segments, and use other directories (maybe residing in a tmpfs too) for temporary files. Analogy: storing man pages in /usr/bin. The system will not crash. It's just bad practice, but you won't find any article on the internet explaining why. The only explanation is that /usr/bin is supposed to contain executable program files, and /dev/shm is supposed to contain named POSIX shared memory segments.