shared memory discussion in detail

sumitfans · 05-09-2009, 04:57 AM

Hi All,

I am currently working on shared memory in Unix.

I have started a process (let say process A) having a int main() and this process will use a key value of key_t key = 1000 for getting a shared memory id (shmid). Once i get the shmid, i attach my data structure and write into that memory.

Then following this i read back from the same memory. The things are working fine.

Now what am I looking for is:

I wont removed that piece of shared memory after exiting from process A. Then I will pass the same shmid to a new process. That new process (let say process B) will attach the same type of data structure using the shmid.

Will process B able to take the content of the memory which process A wrote before exiting ?
Does the memory (shared memory) contains the data even after the process exits / terminates, so that the other process after sometime can go and access the same data ?

Kindly let me know the answers.

Thanks

wje_lq · 05-09-2009, 05:30 AM

Quote:

Originally Posted by sumitfans

Will process B able to take the content of the memory which process A wrote before exiting ?
Does the memory (shared memory) contains the data even after the process exits / terminates, so that the other process after sometime can go and access the same data ?

Yes. The data remains until you use shmctl() to IPC_RMID (remove) the shared memory segment, or until you reboot the machine, whichever occurs first.

sumitfans · 05-09-2009, 05:58 AM

thanx for your response.

let me explain how am I doing things here.

I have a program "test1.c"
In this program I do the following.
1. Create a shared memory using a unique key as 1000.
2. use shmget to get the shmid.
3. attach a pointer to that memory using shmat.
4. write into that memory.
5. detach the pointer from that memory.
6. exit the program using return 0.

Then I run my second executable test2.c by providing the input parameter (argv[1]) as "shmid" which I got by running test1.c program.
In test2.c I try to do the following things.
1. use the shmid and try to attach to the same memory using the pointer of same data type.
2. try to read the memory location.

But this fails and the value in the memory location is set to 0.

I read in some tutorial that once the program exits, the shared memory segment is destroyed.

Is it correct?

wje_lq · 05-10-2009, 05:23 AM

Quote:

Originally Posted by sumitfans

I read in some tutorial that once the program exits, the shared memory segment is destroyed.

I would really, really like the URL of that tutorial! ;)

What do you get when you run the following shell script? I get output something like this:

Code:

generated random number 898425562
found                   898425562

or this:

Code:

generated random number 484319990
found                   484319990

or this:

Code:

generated random number 2095304199
found                   2095304199

Here's the shell script:

Code:

cat > 1.c <<EOD
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <unistd.h>

int main(void)
{
  int   identifier;
  int   int_data;

  int  *int_pointer;

  void *common_pointer;

  identifier=shmget(1000,sizeof(int),IPC_CREAT|0700);

  if(identifier==-1)
  {
    perror("shmget()");

    return 1;
  }

  common_pointer=shmat(identifier,NULL,0);

  int_pointer=common_pointer;

  srand(time(NULL)^getpid());

  int_data=rand();

  printf("generated random number %d\n",int_data);

  *int_pointer=int_data;

  return 0;

} /* main() */
EOD
gcc -Wall -Werror 1.c -o 1
if [ $? -ne 0 ]
then
  echo failed compiling 1.c
  exit
fi
./1
if [ $? -ne 0 ]
then
  echo failed running 1
fi

cat > 2.c <<EOD
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdio.h>

int main(void)
{
  int   identifier;
  int   int_data;

  int  *int_pointer;

  void *common_pointer;

  identifier=shmget(1000,sizeof(int),0700);

  if(identifier==-1)
  {
    perror("shmget()");

    return 1;
  }

  common_pointer=shmat(identifier,NULL,0);

  int_pointer=common_pointer;

  int_data=*int_pointer;

  printf("found                   %d\n",int_data);

  return 0;

} /* main() */
EOD
gcc -Wall -Werror 2.c -o 2
if [ $? -ne 0 ]
then
  echo failed compiling 2.c
  exit
fi
./2
if [ $? -ne 0 ]
then
  echo failed running 2
fi

sumitfans · 05-11-2009, 11:56 AM

hey,
u are absolutely correct

thank you very much !!!

the value is retained even after the program exits. so how long the value remains? is it till we dont restart the system (as u told before)?

also one doubt ... if we detach the pointer in 1.c after writing into the shared memory, will the memory content still be there? if yes then what is the use of detaching from the memory?

and from where the OS gives us the memory. i mean is it from stack or heap?

is the memory contiguous? and what will happen if we request for huge memory?

thanx for your patience ... i know m asking too many questions, but this will clear my doubts ...

sumitfans · 05-11-2009, 12:01 PM

also i observed one more behavior ...

shmid for 1.c program and 2.c programs are different even if the key is same. does this mean that OS gives shmid which is stored as hash value. whichever is first free is being given and hence key is the only identifier between processes to access a shared memory ...

wje_lq · 05-11-2009, 07:31 PM

Quote:

Originally Posted by sumitfans

hey,
so how long the value remains?

As I said before.

Quote:

Originally Posted by sumitfans

if we detach the pointer in 1.c after writing into the shared memory, will the memory content still be there?

It depends on what you mean by "there". After detaching, you can no longer count on the pointer to point to the data. But if you do another shmget(), you'll be able to access the data again, unless IPC_RMID has caused the data to go away.

Quote:

Originally Posted by sumitfans

what is the use of detaching from the memory?

If someone does an IPC_RMID, the actual data is maintained by the operating system until the final detach. So detaching is a way of telling the operating system, "This process doesn't need the data any more." So if some process has done an IPC_RMID, and all processes detach the data, the operating system can reclaim the resources associated with this shared memory and use them for something else.

Quote:

Originally Posted by sumitfans

and from where the OS gives us the memory. i mean is it from stack or heap?

Neither, since the stack and heap are associated with a particular process, and the shared memory can be shared between processes. Exactly how the operating system allocates this memory and keeps track of it depends on the operating system itself, and the application programmer should not concern himself with how this is done.

Quote:

Originally Posted by sumitfans

is the memory contiguous?

Within any one shared memory segment, yes.

Quote:

Originally Posted by sumitfans

what will happen if we request for huge memory?

Try it! You'll see an error return from shmget(), with errno set to ENOSPC.

Quote:

Originally Posted by sumitfans

i know m asking too many questions

That is incorrect.

wje_lq · 05-11-2009, 07:33 PM

Quote:

Originally Posted by sumitfans

does this mean that OS gives shmid which is stored as hash value.

The answer may vary from operating system to operating system, and the application programmer should not concern himself with this question.

Quote:

Originally Posted by sumitfans

key is the only identifier between processes to access a shared memory ...

That is the way you should use shared memory, yes.

sumitfans · 05-12-2009, 11:08 AM

[QUOTE=wje_lq;3537645]Neither, since the stack and heap are associated with a particular process, and the shared memory can be shared between processes. Exactly how the operating system allocates this memory and keeps track of it depends on the operating system itself, and the application programmer should not concern himself with how this is done.

Thanks for your response

I accept that but in my application (which has its own memory manager) i need to know where to allocate a memory from. so if we take Linux OS as an example, does the shared memory is taken from stack or heap?

Also does the shared memory remains in RAM always or else are pages are swapped out and kept in secondary memory?

sumitfans · 05-12-2009, 11:11 AM

Quote:

Neither, since the stack and heap are associated with a particular process, and the shared memory can be shared between processes. Exactly how the operating system allocates this memory and keeps track of it depends on the operating system itself, and the application programmer should not concern himself with how this is done.

Thanks for your response
I accept that but in my application (which has its own memory manager) i need to know where to allocate a memory from. so if we take Linux OS as an example, does the shared memory is taken from stack or heap?

Also does the shared memory remains in RAM always or else are pages are swapped out and kept in secondary memory?

wje_lq · 05-12-2009, 12:09 PM

If your application has its own memory manager, you won't be able to rely on that memory manager to create memory that's shared with another process in the same way that POSIX shared memory works. You'd have to find some other mechanism; a file, perhaps? If that's what you do, then for each process you can put that memory either in the stack or on the heap, but I'd recommend the heap, partly because it's easier to do and partly because stack memory is more limited than heap memory.

And I'm fairly certain that POSIX shared memory can be swapped out like any other memory.

ta0kira · 05-12-2009, 09:59 PM

Quote:

Originally Posted by sumitfans

the value is retained even after the program exits. so how long the value remains? is it till we dont restart the system (as u told before)?

This is actually one of the strongest points against traditional shared memory, in my opinion. Also remember that you can only store self-contained structures in shared memory, i.e. no pointers or structures with pointers.
Kevin Barry

jleslie48 · 01-06-2010, 03:59 PM

hey all,

just stumbled onto this thread looking for an example of shared memory.

this one is beautiful, it works perfectly. Thanks so much for posting it and also the wonderful distribution method!!!

I did have a few questions (probably the answers are there if I RTFM, but my installation of Debian linux doesn't even come up with man pages for shmat or shmget...)

Anyway, can somebody please explain the details, of the parameters of the shmget and the shmat, It looks like the shmget of 1.c made a shared memory area (tag id 1000????)
and 2.c latched onto it with its shmget call, there's something going on with that last parameter to shmget if someone could fill in the details I'd appreciate it.

also both 1.c and 2.c called: common_pointer=shmat(identifier,NULL,0);

so I'm assuming that is the pointer to the shared block of memory? If that could be explained and why shmat has such boring parameters...)

also I'm not sure about the cleanup, as a result of ./1 I created a shared memory section of 4 bytes. I can re-run ./2 as many times as I want, and see that the value is still out there. How do I "see" shared memory sections, something similar to how "ps -ef" show all the pids in use, I would like to show all the made up "shared memory" sections. also what would be the equivalent of the "kill <pid>" of a shared memory, aka, how do I remove the 4 bytes memory so that a run of ./2 will report the shmget error?

and finally although I think I got it, to expand this example to something a bit more complex, the shmget second parameter, would be very well served if it was the sizeof(xyz_type) where xyz_type was a structure of some sort with a whole bunch of stuff that 1.c and 2.c would really want to pass back and forth, yes?

Tia,

Jleslie48

wje_lq · 01-06-2010, 09:28 PM

Quote:

Originally Posted by jleslie48

probably the answers are there if I RTFM, but my installation of Debian linux doesn't even come up with man pages for shmat or shmget

No excuse. Follow ---> this <--- link.

Quote:

Originally Posted by jleslie48

I'm not sure about the cleanup, as a result of ./1 I created a shared memory section of 4 bytes. I can re-run ./2 as many times as I want, and see that the value is still out there.

That will be true until:

your program removes that shared memory segment;
you remove that shared memory segment at the command line with the ipcrm command; or
your system reboots.

Quote:

Originally Posted by jleslie48

How do I "see" shared memory sections, something similar to how "ps -ef" show all the pids in use, I would like to show all the made up "shared memory" sections.

The ipcs command will do this for you.

Quote:

Originally Posted by jleslie48

the shmget second parameter, would be very well served if it was the sizeof(xyz_type) where xyz_type was a structure of some sort with a whole bunch of stuff that 1.c and 2.c would really want to pass back and forth, yes?

Yes. That's how you'd typically use it.

jleslie48 · 01-06-2010, 09:40 PM

Quote:

Originally Posted by wje_lq

No excuse. Follow ---> this <--- link.

That will be true until:

your program removes that shared memory segment;
you remove that shared memory segment at the command line with the ipcrm command; or
your system reboots.

The ipcs command will do this for you.

Yes. That's how you'd typically use it.

hey thanks for the answers. I promise. I will RTFM before I post any followup questions.