LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 03-02-2015, 02:58 PM   #1
multiplex22
Member
 
Registered: Dec 2014
Location: ny, us
Distribution: most
Posts: 56
Blog Entries: 1

Rep: Reputation: Disabled
fork and shared memory


Ok, good one. Like in clone, I want to share memory between forked processes. I can use shared memory calls from shmat, etc. Heres the clincher.

I create CLIST * as a shmget, shmat. The fuctions that allocate memory in CLIST do not. The result is memory in data members were not shared...

Are there other ways? Id hate to convert every library I have to shared memory, simply because of fork. Theres got to be a way.

Last edited by multiplex22; 03-02-2015 at 02:59 PM.
 
Old 03-02-2015, 05:04 PM   #2
mina86
Member
 
Registered: Aug 2008
Distribution: Debian
Posts: 517

Rep: Reputation: 229Reputation: 229Reputation: 229
Use threads?
 
Old 03-02-2015, 07:27 PM   #3
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,679
Blog Entries: 4

Rep: Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947Reputation: 3947
"Show us [a relevant snippet of ...] your code," please. Provide us with a more complete description of your situation, and of your dilemma.

As you know, "fork()" creates an entirely new process, sharing nothing with its parent but a link to the code-segment and a snapshot of the local variables. But, obviously, you know all that. So, I think we can dispense with all of those preliminaries ...

So ... why does your particular situation (whatever it is ...) prompt you to "use fork" and then to "share memory between" them? "What's coming down here, friend? Where are you stuck? What are you trying to do?" Second pair of (been-there, done-that) eyes, at your service ...

Last edited by sundialsvcs; 03-02-2015 at 07:28 PM.
 
Old 03-03-2015, 03:13 AM   #4
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,875
Blog Entries: 1

Rep: Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871Reputation: 1871
I have to admit I don't know what do you mean by CLIST. Would you mind starting your question at the very beginning, not in medias res.
 
Old 03-03-2015, 03:54 PM   #5
multiplex22
Member
 
Registered: Dec 2014
Location: ny, us
Distribution: most
Posts: 56

Original Poster
Blog Entries: 1

Rep: Reputation: Disabled
CLIST is simply what it suggests. Its a dynamic allocated chain array with next pointers instead of static variables.

I want portability on any system. Including Windows. I could use the clone fuction, but this is Linux specific.

I am talking threads, but I want portability. The fuctions in Linux often vary greatly than Windows. Threads are forks, suggested by docs, but indeed copy even the dynamic allocations!

In both processes after a fork, I have the same dynamic memory address using printf. To my surprise, after fork, I change the allocated string to a new statement. Both are to print afterwords. What happens? The child fork prints the original string, while the parent prints the changed string. But wait! They both have the same memory address!

The result is that the child receives no updates from the parent. Only pipes and shmget memory can be used. Even simple malloc data is meaningless. However, what is going on that makes clone differ in regards of sharing data?

Im overall intrigued that mallocs are even copied. However, how do both have the same memory address in dynamic allocations?

Last edited by multiplex22; 03-03-2015 at 04:02 PM.
 
Old 03-03-2015, 03:59 PM   #6
genss
Member
 
Registered: Nov 2013
Posts: 744

Rep: Reputation: Disabled
Quote:
Originally Posted by multiplex22 View Post
CLIST is simply what it suggests.

...

I am talking threads, but I want portability. The fuctions in Linux often vary greatly than Windows. Threads are forks, suggested by docs, but indeed copy even the dynamic allocations!
it doesn't suggest nothing, although google spits out this

(p)threads in linux are made with clone() and locking is implemented with futex-es and some userspace things
 
Old 03-03-2015, 04:19 PM   #7
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,786

Rep: Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083Reputation: 2083
Quote:
Originally Posted by multiplex22 View Post
CLIST is simply what it suggests. Its a dynamic allocated chain array with next pointers instead of static variables.
Never heard the term "chain array", is it like a linked list?

Quote:
I want portability on any system. Including Windows. I could use the clone fuction, but this is Linux specific.
You can't "share memory between forked processes" in a portable way. Windows doesn't even have the concept of "forked processes".

Quote:
I am talking threads, but I want portability. The fuctions in Linux often vary greatly than Windows. Threads are forks, suggested by docs, but indeed copy even the dynamic allocations!
There is a pthreads implementation for Windows. https://www.sourceware.org/pthreads-win32/

Threads are not forks, I wonder which docs you're reading...

Quote:
In both processes after a fork, I have the same dynamic memory address using printf. To my surprise, after fork, I change the allocated string to a new statement. Both are to print afterwords. What happens? The child fork prints the original string, while the parent prints the changed string. But wait! They both have the same memory address!
Virtual memory.
 
Old 03-04-2015, 08:11 AM   #8
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,883
Blog Entries: 13

Rep: Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931
Where am I missing this? I thought the SHMEM utilities were exactly for this where you have separate processes versus what you get with a clone or a thread. You allocate a shared memory segment, you get an ID, that ID is what you share between processes so that they can obtain a semaphore and get ownership and then write data to the shared memory segment. This is exactly what you use it for.

This example is two separate programs, but fundamentally it's the same idea, two separate processes http://www.cs.cf.ac.uk/Dave/C/node27...00000000000000
 
Old 03-04-2015, 09:31 AM   #9
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
I am talking threads, but I want portability.
If multiple threads (rather than multiple processes) is a better fit for your requirements, then use multiple threads.

If there is some portability issue with native threading (though that is hard to understand compared to the greater portability issues of using multiple processes) use some more portable version of multi-threading, such as OMP or Intel's (free and largely portable) TBB.
 
Old 03-04-2015, 09:31 AM   #10
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by multiplex22 View Post
The result is that the child receives no updates from the parent. Only pipes and shmget memory can be used. Even simple malloc data is meaningless. However, what is going on that makes clone differ in regards of sharing data?
Your description is too vague for me to make a decent guess about what misconception is behind your confusion.

Memory can be explicitly shared between processes (in Linux or Windows) so changes written to memory in one process will be readable from memory in the other.

But fork must act as if it copied (rather than shared) all the private (not explicitly shared) portions of the address space. In fact fork really shares, rather than copying almost all the memory, but using a hidden "copy on write" kind of sharing. So that any time either process writes to any page (aligned 4KB chunk) of memory that has the hidden sharing, the kernel rips that sharing apart and gives the writing process its own private copy of the page.


Quote:
Im overall intrigued that mallocs are even copied. However, how do both have the same memory address in dynamic allocations?
That misconception is more obvious. You are apparently unaware about the relationship between virtual addresses and physical addresses. Every address in a process is a virtual address. Every reference to memory goes through a mapping table that converts virtual addresses to physical addresses. Each process has its own mapping table to convert virtual to physical.

On fork, those mapping tables go through the same kind of "copy on write" hidden sharing as the memory itself. As long as an aligned set of 512 pages are shared between two processes, that part of the mapping table could be shared. As soon as any one of those 512 pages is unshared (by copy on write) the page of mappings must also be unshared.

Regardless of those subtleties in fork, a virtual address in one process is not connected to the same virtual address in another process. Linux can make those two addresses map to the same place or to different places within the physical address space.

Last edited by johnsfine; 03-04-2015 at 10:44 AM.
 
Old 03-05-2015, 03:35 PM   #11
multiplex22
Member
 
Registered: Dec 2014
Location: ny, us
Distribution: most
Posts: 56

Original Poster
Blog Entries: 1

Rep: Reputation: Disabled
In my screen shot I have a string with "Original string" in main. I call a fuction that uses fork. The parent returns to main, the child executes recursive printfs. The parent exits first back to main. I call strcpy and set the string to "New string" wait is then called for the child. Child begins executing prints with usleep. Despite parents speed, it prints "Original string"

All string addresses point to the same one in print...

Enc

fork_source.txt
Attached Thumbnails
Click image for larger version

Name:	fork_shared.png
Views:	52
Size:	20.5 KB
ID:	17738  
Attached Files
File Type: txt fork_sorce.txt (3.3 KB, 30 views)
 
Old 03-05-2015, 04:24 PM   #12
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: Fedora
Posts: 4,153

Rep: Reputation: 1265Reputation: 1265Reputation: 1265Reputation: 1265Reputation: 1265Reputation: 1265Reputation: 1265Reputation: 1265Reputation: 1265
Quote:
Originally Posted by multiplex22 View Post

All string addresses point to the same one in print...
No, they don't. Because part of an address is the PID, which specifies what address mapping to use. The PID is different between the two tines of the fork.
 
Old 03-05-2015, 04:48 PM   #13
mdooligan
Member
 
Registered: Feb 2009
Location: Vancouver BC
Distribution: Mandrake10/ArchHackery/Gentoo
Posts: 179

Rep: Reputation: 22
In your sample code all the shm* calls are commented out. Wasn't this a question of shared memory?

Child inherits copies what the parent had at fork(). Even if the addresses *look* the same, they are not. The resemblance is confusing until you understand this. Your eyebrows might look just like your dad's, but they are not your dad's eyebrows.

I'm no guru, but shared memory lives outside any process. You can close the starting process, go make tea, go to work, go on vacation, get home, open a different program 2 weeks later, and link to the shared mem chunk unless someone explicitly closed it or the PC rebooted.
 
1 members found this post helpful.
Old 03-06-2015, 07:06 AM   #14
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,883
Blog Entries: 13

Rep: Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931Reputation: 4931
Quote:
Originally Posted by mdooligan View Post
I'm no guru, but shared memory lives outside any process. You can close the starting process, go make tea, go to work, go on vacation, get home, open a different program 2 weeks later, and link to the shared mem chunk unless someone explicitly closed it or the PC rebooted.
That was my earlier point, explained differently. I like your description.

@OP, I cited a web example, still recommend you give that a look.

IPC: Shared Memory Example two processes comunicating via shared memory: shm_server.c, shm_client.c
 
Old 03-06-2015, 08:10 AM   #15
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Quote:
Originally Posted by multiplex22 View Post
Despite parents speed, it prints "Original string"

All string addresses point to the same one in print...
At some point please take the time to read the answers you already received. Otherwise, what was the point of asking the question.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to create a shared memory and some shared objects in linux? jeremy28 Programming 6 06-11-2010 05:21 AM
Does /dev/shm reduce memory available for non-shared memory segments? mightyscotchpine Linux - Server 1 09-22-2009 06:58 PM
Linux shared memory segment access problem and x86 Virtual Memory layout. regmee Linux - Kernel 1 08-23-2008 12:11 AM
Difference between resident memory,shared memory and virtual memory in system monitor mathimca05 Linux - Newbie 1 11-11-2007 04:05 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:40 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration