LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 03-25-2009, 07:54 AM   #1
kam2630
LQ Newbie
 
Registered: Jan 2008
Posts: 6

Rep: Reputation: 0
Linux Shared Memory Usage


Hi,

I need to write a complex data processing system with linux 2.6.xx, with
consists of different processes. Two of them using a small shared memory region to share sampled data with a rate of 50 msec.
The shared segment will be opened with mmap referenced to a file on a compact flash card.
From the mmap function description I read, that the data written by process A will be seen from process B "immediately" which will be true,
and also that the underlying file (referred fro mmap) will only be written if one of the processes will unmap the shared region or calling msync.

If I do a look onto the filesystem with ls -l on the referred file
of the shared segment, I see that the file file stamp reflects always the actual time, so it seems that the file will be updated continously,
which will be a disaster using a cf card and the necessary sampling rate.

Is there any possibilty to control (disallow) the system to flush the dirty region of the shared segment to the file ?

any help appreciated,
Thanks in advance
 
Old 03-25-2009, 08:12 AM   #2
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 454Reputation: 454Reputation: 454Reputation: 454Reputation: 454
Quote:
Originally Posted by kam2630 View Post
Hi,

I need to write a complex data processing system with linux 2.6.xx, with
consists of different processes. Two of them using a small shared memory region to share sampled data with a rate of 50 msec.
The shared segment will be opened with mmap referenced to a file on a compact flash card.
From the mmap function description I read, that the data written by process A will be seen from process B "immediately" which will be true,
and also that the underlying file (referred fro mmap) will only be written if one of the processes will unmap the shared region or calling msync.

If I do a look onto the filesystem with ls -l on the referred file
of the shared segment, I see that the file file stamp reflects always the actual time, so it seems that the file will be updated continously,
which will be a disaster using a cf card and the necessary sampling rate.

Is there any possibilty to control (disallow) the system to flush the dirty region of the shared segment to the file ?

any help appreciated,
Thanks in advance
Regarding the item in bold - why do you think so ? I.e. I don't think the file in question belongs to CF filesystem. I.e. I don't think /dev/shm* belongs to your CF filesystem. I.e. you need 'man shm_open'.
 
Old 03-25-2009, 08:37 AM   #3
wje_lq
Member
 
Registered: Sep 2007
Location: Mariposa
Distribution: FreeBSD,Debian wheezy
Posts: 811

Rep: Reputation: 179Reputation: 179
Quote:
Originally Posted by Sergei Steshenko View Post
I don't think /dev/shm* belongs to your CF filesystem.
That's correct, it doesn't.

Quote:
Originally Posted by Sergei Steshenko View Post
man shm_open
That is one way to share memory.

Quote:
Originally Posted by kam2630 View Post
The shared segment will be opened with mmap referenced to a file on a compact flash card.
This is a completely different way.

If there is no real reason to use a CF card, he can either do what he's doing now but using /dev/zero, or he can write completely different code to use shm_open() and friends.

Unfortunately, I have no answer to the original question, but I wanted to clarify the issue of CF flash card versus shm_open().

The only other thing I would suggest is that if there is any way to monitor statistics (perhaps through /proc or something, maybe that would be the way to monitor actual hardware throughput. I don't know, and I'm hoping that someone else will step forward with an answer.
 
Old 03-25-2009, 08:37 AM   #4
kam2630
LQ Newbie
 
Registered: Jan 2008
Posts: 6

Original Poster
Rep: Reputation: 0
Dear Sergei,
thanks a lot, that's what I need.
 
Old 03-25-2009, 09:31 AM   #5
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,671
Blog Entries: 4

Rep: Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945
If you've got a time-intensive requirement for updating data, it puzzles me why you are posting the data to a flash card. That's dependent upon a "physical" I/O operation, to a capacious-but-slow device. (Even if it appears to be "RAM," you know that in order to actually get to it you must perform USB I/O operations... and you know that the cycle-time of the memory on that card is relatively slow.)

I suggest that you simply map a shared memory segment... in RAM. If you need to post the data to a flash-card, dedicate a separate thread to the task of flushing the data to the card.

As I envision it, this separate thread would periodically grab the data from the shared-memory (using a mutex, of course, to assure data integrity), and it would move that "snapshot" off to another memory area for its own use. Having done so, it would release the mutex and then push the "snapshot" out to the card. (This "two-step" arrangement avoids having the mutex remain locked while the I/O is taking place.)

In this way, the data on the card would be reliably kept up-to-date, but none of the data-collecting processes would ever have to wait on it. System performance will remain stable and predictable. The card will not overheat.

Last edited by sundialsvcs; 03-25-2009 at 09:32 AM.
 
Old 03-25-2009, 02:56 PM   #6
kam2630
LQ Newbie
 
Registered: Jan 2008
Posts: 6

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by sundialsvcs View Post
If you've got a time-intensive requirement for updating data, it puzzles me why you are posting the data to a flash card.
I have an embedded device, no mechanical disk, and a pure small system memory.

Quote:
Originally Posted by sundialsvcs View Post
I suggest that you simply map a shared memory segment... in RAM.
Are you expecting a ram disk, or are there any other possibility to
use an RAM based shared segment without file buffering ?
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Linux shared memory segment access problem and x86 Virtual Memory layout. regmee Linux - Kernel 1 08-23-2008 12:11 AM
Difference between resident memory,shared memory and virtual memory in system monitor mathimca05 Linux - Newbie 1 11-11-2007 04:05 AM
usage of shared memory dianarani Linux - Hardware 1 10-16-2007 03:12 PM
X hangs after prolonged usage ... memory usage goes upto 195MB duryodhan Slackware 18 05-10-2007 06:16 PM
how to determine cpu usage, memory usage, I/O usage by a particular user logged on li rags2k Programming 4 08-21-2004 04:45 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration